In December 2008, I wrote a blog post titled 9 Genealogy Predictions for 2009.  It is now time to review that list and see how well the predictions matched reality.

1.  Two more desktop genealogy applications will support source citation templates from Elizabeth Shown Mills’ Evidence Explained. Currently Legacy 7 and RootsMagic 4 support this. The other two might likely be Family Tree Maker and The Master Genealogist.

Family Tree Maker 2009 now supports source citation templates following Evidence Explained.  To my knowledge, no other desktop genealogy applications have announced this support.

2.  One major online database (Ancestry, WorldVitalRecords, FamilySearch, Footnote) will announce upcoming support for Evidence Explained source citations.  Other sites will soon follow with their own announcements.

I am disappointed that none of the mentioned online databases support Evidence Explained source citations.  Please correct me if I am mistaken.  If GenSeek is released in 2010, maybe it will be the first.

3.  FamilySearch Research Wiki will grow to over 30,000 content pages (excluding talk and stub pages) by the end of the year. As of 28 December 2008, there were 12,573 content pages.

On December 28, 2009 there were a total of 20,298 content pages on the FamilySearch Research Wiki.

4.  FamilySearch will announce a replacement for GEDCOM. GEDCOM will still be supported for many years as software will support it as well as the newer format. Reasons for the new format will be to better support source citations and media.

This did not happen in 2009, but it might happen in 2010.

5.  I hope I am wrong about this one, but with the current economic downturn we could see financial pressure force one of the more-recently-started genealogy companies to merge with or be acquired by another company.  Worse case scenario, it could go bankrupt.

To my knowledge this did not happen.  In fact, Ancestry went public in 2009.

6.  Many more genealogy societies will join the social networking site, Facebook, by adding a group page. Searching groups today for “genealogical society | genealogy society” (the pipe symbol “|” means “OR”) returns 80 results. I wouldn’t be surprised if by the end of 2009 the total was over 200.

Finally a correct prediction! Doing a group search on Facebook resulted in 286 results for genealogy societies.

7.  More genealogy applications will be available for the iPod Touch and the iPhone. Currently the iTunes App Store only lists three applications: MobileFamilyTree, FamViewer, and Shrubs.  This last holiday season, the iPod Touch was a hot item.  I wonder how many genealogists own one now.

This one was easy to predict as even one more app would have fulfilled the prediction.  There are currently 7 applications listed in the App Store:

MobileFamily Tree – $4.99

Reunion – $14.99

GedView – $3.99

FamViewer – $14.99

Mobile Tree – $4.99

Shrubs – $9.99

iCensus – $3.99

To learn more about some of these apps, check out the blog post titled, Genealogy Apps for iPhone, by Joan Miller. 

8.  Through blogs and podcasts, genealogists will hear more about the benefits of Web 3.0 (also called the Semantic Web) and how it applies to genealogy.

Web 3.0?  What is that?  You can find discussions about microformats and full fledged ontologies, but not much applied to genealogy in 2009.

9.  Genealogists are puzzle solvers when it comes to researching their family history, but do they play games?  I expect to see a genealogy-related game released in 2009.  It could be a casual game for Facebook or something more interactive for the Wii.  How about a Mii Familii Trii?  The game might teach research-related concepts (citation, transcription, etc) or (via GEDCOM import) put your ancestors in the game.

There is still so much opportunity in this area without much action.  I did find a casual game called Family Mystery: The Story of Amy where genealogy is used as the theme for the game but not much else.

So out of 9 predictions, I give myself a score of 3.5 which is about as often as I was correct when as a teenager I operated the “guess your age or weight” game at an amusement park.  After a few days, I was moved to the skee ball games which fit my talents much better.

Best wishes for a wonderful 2010 full of genealogy innovation and hopefully some surprises!

Source, Information, & Evidence

According to Evidence Explained by Elizabeth Shown Mills, ”sources are artifacts, books, digital files, documents, film, people, photographs, recordings, websites, etc.” (see page 24)  Information is the content of the source.  Evidence “represents our interpretation of information we consider relevant to the research question or problem.” (see page 25)  So in order to classify evidence we need both information and a research objective.  Even though the Genealogical Proof Standard (GPS) does not include a step to define research goals, I’ve included it as part of the Genealogy Research Process Map because it is implied.  Step one of the GPS states:

“We conduct a reasonable exhaustive search in reliable sources for all information that is or may be pertinent to the identity, relationship, event, or situation in question.”
The BCG Genealogical Standards Manual, page 1.

How do we know which sources to search if we don’t have a research objective?  The definitions of direct and indirect evidence also points to the need to have a defined research objective:

Direct evidence – relevant information that seems to answer the research question or solve the problem all by itself.
Indirect evidence – relevant information that cannot, alone, answer the question;
Negative evidence – an inference we can draw from the absence of information that should exist under particular circumstances.
Evidence Explained, page 25

Even the definition for negative evidence hints at a research objective.

So how can we set the citation quality value for evidence in RootsMagic or any other genealogy software unless we have a research objective?

A research objective can be in the form of a question, statement, or hypothesis.  The following are all valid research objectives:

• What was Worth Tucker’s birth date and place?
• Where was Worth Tucker born?
• Worth Tucker was born 30 Nov 1870 in Ashe County, North Carolina.
• Was Worth Tucker born in 1869 or 1870?

How specific the research goal depends on the researcher and what they want to discover.

In RootsMagic, a source can be associated with a person, family, or fact:

And for each citation of a source you can specify a value for citation quality for source, information, and evidence.  But to classify evidence, what is the research objective implied or otherwise?

Let’s say we have a book that we use as a source.  It was created by its author and is an original source.  The source information is entered in the Master Source section and appears in the Master Source List.  A specific page of that book is specified in the Source Details section and some information from that page is entered as part of the Source Detail Text.  This information then needs to be evaluated based on who provided that information: the informant.  If the informant was a participant or eyewitness, then we can say that the information is primary, otherwise it is secondary.  Now we come to evidence.  We take that piece of information and compare it to our research objective, but since we don’t have one we must infer it from what the citation is related to: person, family, or fact.  It is important to note that it is usually the researcher that should determine the research objective and not the software. 

Since we don’t have research objectives, only what the citation is associated with, then we must imply the research objective.  This can be tricky. 

Evidence for Fact Citation

Since a birth fact allows you to enter date, place, and place details does information in the source we are citing need to contain all of this information before we can say that it is direct?  Put another way, is the implied research objective:

What is the date, city, county, state, and specific place of Worth Tucker’s birth?

or it is

What is the date and place of Worth Tucker’s birth?

Do they both mean the same thing?  There is room for ambiguity.  Not to mention the fact that as a researcher, maybe I wanted the research objective to be a statement or a hypothesis:

Worth Tucker was born 30 Nov 1870 in Ashe County, North Carolina.

Depending on the fact, the software ask us to enter different information:

• Birth – date, place, place details
• Alternate Name – given names, surname, prefix, suffix, name type, nickname, date
• Education – date, place, place details, description
• DNA – DNA results
• LDS Baptism – date, temple, status, live/temple

One workaround for fact-based citations is to define a guideline that the citation quality can be marked as direct only if all possible entry fields for that fact type are filled in completely.  If a date or place is only partially filled in or the information doesn’t allow all fields to be filled in, then it is indirect.

Evidence for Person Citation

What would be the implied research objective for a citation associated with a person?

When you add a new person it asks for given name(s), surname, prefix, suffix, sex, and nickname plus some vital facts.  So I guess the workaround for this situation is to have the research objective be:

What is the complete name, sex, and optional nickname of Worth Tucker?

If the information from the source can answer all the parts, then it is direct; otherwise it is indirect.

Evidence for Family Citation

Now what about the implied research objective for a citation associated with a family?

What are the complete names of both parents and all children as well as the birth order of the children for the marriage of Worth Tucker and Florence Abby Pulsipher?

If the information in the source can answer all parts of the question, then the evidence is direct.

An Eye to the Future

Part of the reason for the current implementation in RootsMagic 4 and similar software could be the decades-old focus on research results and not research analysis.  My first genealogy software was a DOS version of PAF.  Before using that software, all research and any conclusions I came up with were done on paper.  Even the first versions of PAF only allowed you to enter a single date for birth, christening, death, and burial.  So the software was intended only to record the results of your research.  Modern genealogy software allows you to enter multiple facts/events of the same type and associate a source citation to it.  That way, you could have half a dozen birth entries containing different values or different levels of detail as the your research required. The software is able to keep track of ongoing research, and not just results.  I think genealogy software will continue to innovate and allow more and more research collection and analysis to occur in the software.  That is the direction that citation quality is pointing in RootsMagic 4 and the similar feature in Family Tree Maker 2009.  The next logical step is to allow the software to define research projects each with their own research objectives.  These research objectives would then be associated with source citations and citation quality therefore clarifying the use of the classification of evidence as direct or indirect.

I strongly encourage users of RootsMagic to use this feature, but in its current implementation there are a few gotchas and workarounds that need to be followed.

The Genealogical Proof Standard & Evidence Explained define research analysis classifications for a source, information, and evidence. A source is an object (or person) that contains (or has) information. A source can be classified as original or derivative. An original source is in its first oral or recorded form. Everything else that comes from an original (or another derivative) is a derivative. For example, a book is an original. Let’s say that it is a census enumerator’s book that he carried from house to house to take the census. Now let’s say that book is microfilmed and stored at an archive. The microfilm copy is a derivative. The digitization of the microfilm is a second generation derivative of the original. Without getting into the special cases of image copies, duplicate originals, and record copies, it is relatively easy to start uncovering the provenance or ancestry of the source you are using for your research back to the original source. The classification of a source as original or derivative helps to answer the question “Is there a better source?” and helps in your analysis as original sources usually carry more weight than derivative.

In RootsMagic 4 on the Edit Source screen, you see both the Master Source and Source Details sections.  The information in the Master Source section is independent of the details recorded when a specific source is cited for a person, family, or fact. 

When you click on the Quality button you see the three categories for sources, information, and evidence with their possible values:

Mapping this screen to the previous, the Source category refers to the Master Source section and the Information category refers to the Source Details section.  I’ll discuss the Evidence category in another gotcha.  What this is saying is that a source is original or derivative independent of the information that it contains.  The book, A History of Emery County, is an original source written by Edward A. Geary and it remains an original regardless of the degree of knowledge (firsthand or secondhand) of the informant that provided the information contained on page 179 or any other page. I can understand putting all three categories on the same screen to simplify things for the genealogists using the software.

In the current implementation in RootsMagic 4 (version 4.0.3.0) the problem is that all three categories are associated with the Source Details section.  This is a problem because this source appears in the Master Source List and can be used to cite multiple sources:

What that means is that if you set the Source as Original for the first citation, it will not be reflected in subsequent citations.  Creating a second citation from that same Master Source List entry will default the Source to “Don’t know” and it must be set independently.  Let’s suppose you have 10 citations (different Source Details) for a single Master Source List entry.  You could have 5 set to Original, 3 to Derivative and 2 to Don’t know.  And if you change the value for one citation it is not reflected in the others.  Worst case, this is confusing plus requires extra work to make sure all of the sources are consistent.

The current work around is to add a Master Source Comment indicating that this source is original or derivative and then each time you cite this existing source, check the comment before setting the quality classification for Source.

It is my hope that in a future release, there will be a way to set the source as original or derivative from the Edit Source screen off the Master Source List:

Then when that source is cited again the Quality screen would default the Source to that same value.  The Source in the Quality screen could be read-only or if editable any changes would be reflected in all citations derived from that same master source.

I hope this constructive criticism is taken how it is intended: to make an important feature the best that it can be.

As early as the 1997 book, Evidence! Citation & Analysis for the Family Historian, Elizabeth Shown Mills has covered the topics of citation and analysis.  It is this second item, analysis, that is the focus of this innovator award. In Evidence! we start to see the formation of the current classification for sources (as original or derivative) and evidence (as direct or indirect).  The BCG Genealogical Standards Manual published in 2000 supports the classification of sources (as original or derivative), adds a classification for information (as primary or secondary), and continues the classification of evidence (as direct or indirect).  These classifications remained unchanged in Professional Genealogy which was published in 2001.  By 2006 as seen on quick sheet, Evidence Analysis: A Research Process Map by Elizabeth Shown Mills we see the formation of a new evidence classification so in addition to direct or indirect we can classify evidence as negative evidence.  When Evidence Explained was published in 2007 it restated these same classifications for sources (original or derivative), information (primary or secondary), and evidence (direct, indirect, or negative).

The winner of the next innovator award not only supports Evidence Explained citation templates but has coded these professional analysis practices into their software in a way that is approachable to all.  So a big congratulations goes out to Bruce Buzbee and his RootsMagic team!

Let’s look at the implementation in more detail.

In RootsMagic 3, we had a way to specify the quality of a source but the feature in RootsMagic 4 is dramatically improved.

Here is the citation quality dropdown list from the Edit Citation Details screen in RootsMagic 3:

The list contains primary, secondary, questionable, and unreliable.

Compare that to the citation quality screen from RootsMagic 4:

As you can see, RootsMagic 4 supports separate classifications for source, information, and evidence and fully supports these classification following Evidence Explained.

You get to this screen by clicking the Quality button on the Edit Source screen:

The next closest competitor of this feature is Family Tree Maker 2009, but the software fails to acknowledge the existence of negative evidence:

I am not sure when each product officially released this feature. The best I can estimate is that Family Tree Maker 2009 was released the first of February 2009 and RootsMagic 4 the end of March 2009.  I was aware of this feature in RootsMagic 4 all the way back in July 2008.  Since RootsMagic 4 handles negative evidence, it gets the award.

I feel that this is a transformational innovation because as more genealogists and family historians at all skill levels use this common terminology, analysis will become more consistent and communication will become more clear.  Look for other genealogy software (desktop as well as online) to support citation quality following the genealogical standards in Evidence Explained in the near future.

There are some changes and additonal improvements that I would to see in the area of citation quality that I would like to address in future posts. This is truly an exciting time.  Genealogy software is moving more towards helping with reserach analysis and not just the recording of research results.  Thanks RootsMagic for being a leader in this area.

To see other ThinkGenealogy Innovator award winners, visit the winners page.

In this post, we continue our exploration through existing bibliographic standards to see how they might work as a format for online sites to easily share citation information.  To see the journey we have made so far, visit the page, A Better Way to Cite Online Sources.

From the Library of Congress standards page for MODS, we see the following description:

Metadata Object Description Schema (MODS) is a schema for a bibliographic element set that may be used for a variety of purposes, and particularly for library applications.

On the MODS overview page, we get more details:

As an XML schema it is intended to be able to carry selected data from existing MARC 21 records as well as to enable the creation of original resource description records. It includes a subset of MARC fields and uses language-based tags rather than numeric ones, in some cases regrouping elements from the MARC 21 bibliographic format. This schema is currently in draft status…
…the schema does not target round-tripability with MARC 21. In other words, an original MARC 21 record converted to MODS may not convert back to MARC 21 in its entirety without some loss of specificity in tagging or loss of data. In some cases if reconverted into MARC 21, the data may not be placed in exactly the same field that it started in because a MARC field may have been mapped to a more general one in MODS.

Compared to MARC, MODS is simplier and uses word tags (like name, titleInfo, and originInfo) instead of numeric tags (100, 245, 260).  There is not a 1 to 1 mapping between MARC and MODS, so conversion between the two might introduce some challenges.

Let’s look at the book example used in the analysis of the other standards:

Geary, Edward A. A History of Emery County. Salt Lake City: Utah State Historical Society, 1996.

The Library of Congress represents this book in MODS here.

The three key pieces of information (author, title, and publication) are represented in MODS as follows:


<titleInfo>
  <nonSort>A </nonSort>
  <title>history of Emery County</title>
</titleInfo>
<name type="personal">
  <namePart>Geary, Edward A.</namePart>
  <role>
    <roleTerm type="text" authority="marcrelator">creator</roleTerm>
  </role>
</name>
<originInfo>
  <place>
    <placeTerm type="text">Salt Lake City</placeTerm>
  </place>
  <publisher>Utah State Historical Society</publisher>
  <dateIssued>1996</dateIssued>
</originInfo>


Especially compared to MARC, this is much easier to understand. The title is shown between the titleInfo elements and splits the title into the non-sorting (/filing) portion and the rest of the title. The author’s name is contained in the name section identifying it as a personal name with the actual name contained in the namePart element.  The role\roleTerm is used to identify that the name is the creator of the source which by implication for a book is its author.  The originInfo section includes the publication information including publication place (place\placeTerm), publisher, and publication date (dateIssued).

Something else I noticed about this compared to MARC is that extra punctuation (commas, periods, colons, and semicolons) are not included as part of the field values. This separates the formatting from the data which makes more sense to me.  I also like the ability of MODS to break out the name and describe the individual parts:


<name type="personal">
  <namePart type="given">Edward A.</namePart>
  <namePart type="family">Geary</namePart>
</name>

 This can give more flexibility when displaying the name in the Source List Entry (surname first), First Reference Note (given name first), and Subsequent Note (surname only).

When citing a book with multiple authors and those authors have a name prefix (ex: Dr.) and/or a name suffix (Jr., III, etc.) is becomes necessary to break out those name parts as well to aid in forming the citation.  This can be done in MODS as follows:


<name type="personal">
  <namePart type="termsOfAddress">Dr.</namePart>
  <namePart type="given">Benjamin Franklin</namePart>
  <namePart type="family">Gates</namePart>
  <namePart type="termsOfAddress">III</namePart>
</name>


Notice how both the name prefix and suffix are identified by the same namePart type attribute of “termsOfAddress”.  In MODS, the order of the elements does not imply any display order so for a computer program to know that the first termsOfAddress is a prefix and the second is a suffix it would have to parse the actual text value.  In my opinion, it would be more helpful if MODS identified them like this:


<name type="personal">
  <namePart type="prefix">Dr.</namePart>
  <namePart type="given">Benjamin Franklin</namePart>
  <namePart type="family">Gates</namePart>
  <namePart type="suffix">III</namePart>
</name>


There is more investigation of MODS still to be done to determine how it might represent additional citation details that isn’t usually found in the Source List Entry. Remember page number for book? Additional sample citations from Evidence Explained would need to be put to the test to see if MODS is a good fit. I do like the ability of XML to add additional namespaces to qualify elements and attributes. This could be a means of extending the MODS standard for any additonal details required.

Previous posts have explored a better way to cite online sources (Part 1), how citation information can be stored as a file using GEDCOM format (Part 2) and MARC format (Part 3). This post takes the next logical step and discusses MARC XML.

MARC was created as a machine-readable format many decades ago. In the last decade, eXtensible Markup Language (XML) has been developed as a standard format to allow validation, processing, and transformation of data. MARC XML takes the MARC format and represents it as XML. This is done in a lossless way so that conversions between MARC and MARC XML will not lose any data.

A book represented as a Source List Entry in Evidence Explained looks like this:

Geary, Edward A. A History of Emery County. Salt Lake City: Utah State Historical Society, 1996.

That same book listed with the Library of Congress is shown here as MARC XML.

Let’s quickly compare the MARC entries for author, title, and publication with the corresponding representation in MARC XML.

MARC

100 1#
  $a Geary, Edward A.,
245 12
  $a A history of Emery County.
260 ##
  $a Salt Lake City :
  $b Utah State Historical Society ;
  $c 1996.


MARC XML

  <datafield tag="100" ind1="1" ind2=" ">
    <subfield code="a">Geary, Edward A.,</subfield>
  </datafield>
  <datafield tag="245" ind1="1" ind2="2">
    <subfield code="a">A history of Emery County.</subfield>
  </datafield>
  <datafield tag="260" ind1=" " ind2=" ">
    <subfield code="a">Salt Lake City :</subfield>
    <subfield code="b">Utah State Historical Society ;</subfield>
    <subfield code="c">1996.</subfield>
  </datafield>


The three fields (author, title, publication) are each represented by a datafield element with corresponding tag and indicators (ind1, ind2).  The only difference being in my reformatting of MARC, indicator spaces were represented by the # sign.  Instead of the $ delimiter for subfields, each datafield element has a separate subfield element with the appropriate code letter.

The MARC XML representation is easier to read by a person (software developer) than the raw MARC file and is more easily processed due to most programming languages’ ability to work with XML.  XML is more descriptive and therefore file sizes would be larger for MARC XML than for MARC.  But the size is not likely to be an issue.

Because MARC XML is just a rendering of MARC into XML, the observations and questions from Part 3 apply.

MARC XML could also be a possible format to represent online source citation information.

Between the script and the detailed description of the demo, you should be in a good position to answer the survey questions without the need to see the video.

Take the Survey!

Please take a few minutes to complete a survey about citing online genealogy sources.

• Click Here to take survey for individual genealogists or family historians
• Click Here to take survey for companies and organizations that provide genealogy software or services

SCRIPT:

Sooner or later anyone doing genealogy or family history research will meet up with the challenge of source citations. Some will immediately dismiss the topic and move on. Others will find the topic too big of a challenge to even try. Still others will set out with the best of intentions but will get bogged down in the complexity. The numbers that face the learning curve of source citations and conquer its intimidating slope are relatively few.

This has lead many to believe that citing sources is only for the professional genealogist. You may hear them say: “I’m just a personal family historian, no need to cite my sources.” The last time most of us were asked to create a bibliography or cite a source was for a paper we wrote in high school or college.

Researchers at every level should seek for credibility in their research. How else will we be able to separate fact from fiction? Citations are a minimum for our work to be considered as credible. The question is not “do we need to cite sources?” its “how do we make citing sources achievable by all?”

Currently citing sources is just too difficult. In recent years this is getting easier with the creation of genealogy citation guides, quick sheets, and software. But it needs to be even easier still.

There is another problem when it comes to online genealogy database websites like FamilySearch, Ancestry, Footnote, WorldVitalRecords, GenSeek and others. Similar sources on each of these websites have different citation formats or even no citations at all.

The amount of genealogy information currently on the internet is like a swimming pool full and in just a few years that will grow to an ocean full. To avoid adding more confusion to an already confusing situation, there needs to be standardized citations across these genealogy database websites.

In this demonstration I would like to show how easy it can be to cite online sources in your desktop genealogy software. What I am showing is prototype code. I am using RootsMagic 4 because of its support for citation templates from Elizabeth Shown Mills’ Evidence Explained. Other desktop genealogy applications that support these templates are: Legacy Family Tree 7 and Family Tree Maker 2009.

First I will create a new database in RootsMagic and will go ahead and close it. Now I will visit a sample genealogy website that contains some information on my ancestor, Worth Tucker. This website represents one of the larger online genealogy database sites mentioned earlier or could be a site containing someone’s personal research. On this site we find information about Worth Tucker’s property ownership in Elmo, Emery County, Utah. There is an extract from the book, A History of Emery County, as well as images of the book’s cover, title page, copyright page, and page 179. Notice also that there is a source citation for the book following a format in Evidence Explained.

Because the website contains the citation and supporting documents, I could use my desktop software to create a person entry for Worth and a fact or event entry for property ownership. I would then create a source and detail entry for the citation and link it to the event. I could also download the images to my computer and associate them to the source. But that is even more complicated than it needs to be.

This website supports a “Quick Cite” feature that allows easy import of the citation and associated files into my desktop genealogy software. When I click the button, the information is downloaded to my computer and the import begins. Since I have other genealogy software installed on my computer, I will be asked to select which program to use. In just a few seconds the information will be imported into my software and when I open my database then I will be able to see all the information properly cited.

Here is my entry for Worth Tucker showing the property fact. This fact has one source following the “Book, Basic format” template from Evidence Explained. The template fields are filled out correctly. Also imported was the source media. Here you find the three images for the source and here the image for the page that was cited. It even imported the extracted text from the source.

Now that is the way that citation of online sources should be done!

Is this a feature that interests you?

The technology to do this exists right now.

What is needed is for us to speak up and let the creators of our genealogy software and the online database websites know that this is a feature that we want. When the genealogy community unites to petition for records preservation and access, numbers count. The same is true when we want solutions to difficult challenges like source citation.

Contact the companies and organizations whose products and services you use. Let them know what you want. Send them a link to this video. Share this with friends and associates via e-mail, social networking sites or blogs.

Another way to participate is to complete a 5-minute survey that describes your level of interest in such a feature. The results will be shared with any individual, organization or company that completes the survey.

There is one point of clarification that I would like to make: although the technology exists today to do this type of “one click” citation it has not been implemented. What I showed was a prototype proving that it was possible. What needs to happen next is for online organizations (like Ancestry, FamilySearch, World Vital Records, Footnote, GenSeek, etc.) and genealogy software companies (like the makers of RootsMagic, Legacy, and Family Tree Maker) to agree on a file format and implement it. The online organizations would need to start providing a file with a download link for each source on their site and the genealogy database applications would need to support importing of the files. One could view the video as a tutorial of how things could be. That is precisely why it is so important to get the word out.

This is a grassroots effort from the genealogy community to let genealogy software and service providers know what we want. Citing sources is important to us. There are inconsistencies and other problems that need to be solved. Let’s find a way to get together and solve them. I am just one voice trying to spotlight an issue and provide a way for others to get their voices heard. We have had 200 respondents to the individual survey with
83% classifying themselves as non-professional genealogists.

Here are a few more statistics:

88% of respondents say that citing sources is very important with the remaining 12% saying it is important. 77% say they find citing sources difficult or very difficult. When asked if there should be one standard citation style for genealogy, 90% responded in the affirmative. Respondents used Evidence Explained the most (62%) as a guide for genealogy research. The two most used online genealogy sites are Ancestry.com and FamilySearch. It would be great if one of those would join in support of this effort. When asked “Does the ability to click a link on a website and have the source citation imported into your genealogy software interest you?” 84% said they were very interested with an additional 10% saying they were interested. Instead of 1100 views of the YouTube video, I would like to see 10,000 with at least 1000 filling out the survey. Maybe at that point this effort will have enough momentum to really get moving.  Is that too much to ask from the genealogy community?

One surprising and disappointing statistic is the low numbers taking the company and organization survey; only 4.  All 4 felt that citing sources were very important and all would be willing to participate in the cause.

Once again I want to thank Dick Eastman for his contributions to the genealogy community on this and many other issues.

Keep spreading the word and let your voice be heard.

In Better Online Citations – Details Part 1 we examined how the QuickCheck model for “Book: Basic format” from Evidence Explained was coded in Family Tree Maker 2009, Legacy 7, and RootsMagic 4. From the screens we were able to identify implementation differences between the three applications. There are also differences between the applications in how citation information is conveyed via a GEDCOM export. The individual fields shown on the template screens are lost in the standard GEDCOM export making it impossible to create a rich EE-style citation in one application, export it to GEDCOM, and import it into another application while retaining that richness. In all cases (except when the exporter and importer of the GEDCOM is RootsMagic 4), the citation is changed from a “Book: Basic format” to a generic “old-style” (pre EE) format with important details lost.

In the previous post I skipped all details of the file format needed to support online downloadable source citations. From my own observations, from survey feedback as well as comments on this blog and through e-mail there are two main camps when it comes to this topic. One group feels that the best approach to take is an extension to the existing GEDCOM 5.5 standard (which was released in 1996). They feel that it is the best choice to lower the barrier of adoption and avoids YAFF (Yet Another File Format). See the comments from Tamura Jones from Part 1. The second group is open to a new file format based on XML (Extensible Markup Language) which has wide support among programming languages. As a programmer I lean slightly to the side of XML and that is what I used in the prototype shown in the video. But I am open to either view.

In that spirit of openness, I will first look at how the three applications that support EE-style citations represent those citations in an exported GEDCOM file. In this post, we will look at sections of GEDCOM which is a little technical but there will be plenty of explanations for readers of all levels. From the previous post we saw differences in how each application implemented the EE-style QuickCheck model and this post will show differences in how each exports them. You will also see some forethought in the export from one vendor.

For those who have never seen a GEDCOM file, each line starts with a number: 0, 1, 2, etc. A zero is the beginning of a new record and higher numbers are “nested” under lower numbers. It is a way of grouping information together. In the GEDCOM fragments I show here, I will indent the lines to make them easier to understand. I have also done some slight rearranging of lines to make the comparisons easier. After the number, each line has a tag which is a shortened identifier of the information contained on that line. You can see definitions of the standard tags here.

A GEDCOM file starts with a header that describes the software application that generated the file as well as the version of the GEDCOM standard:

Family Tree Maker 2009

0 HEAD
	1 SOUR FTM
		2 VERS Family Tree Maker (18.0.0.305)
		2 NAME Family Tree Maker for Windows
		2 CORP The Generations Network
	1 DEST GED55
	1 GEDC
		2 VERS 5.5
		2 FORM LINEAGE-LINKED

Legacy 7

0 HEAD
	1 SOUR Legacy
		2 VERS 7.0
		2 NAME Legacy (R)
		2 CORP Millennia Corp.
	1 DEST Gedcom55
	1 GEDC
		2 VERS 5.5
		2 FORM LINEAGE-LINKED

RootsMagic 4

0 HEAD
	1 SOUR RootsMagic
		2 VERS 4.0
		2 NAME RootsMagic
		2 CORP RootsMagic, Inc.
	1 DEST RootsMagic
	1 GEDC
		2 VERS 5.5.1
		2 FORM LINEAGE-LINKED

The first line starts the header record which contains a source program (SOUR) that generated the file and a destination program or format (DEST) that will use the file. The level under source shows the program, version and company information. The GEDC or GEDCOM tag identifies that the file adheres to the specified GEDCOM version and format. FTM and Legacy shows 5.5 whereas RM shows 5.5.1. GEDCOM 5.5.1 was published as a draft in 1999 but the changes between it and GEDCOM 5.5 do not affect our discussion. The rest of the header record was removed as it is not important for our comparison. Even though the numbering and abbreviations take a little getting used to, the file format is pretty straight forward. The rest of the file is just as understandable.

As you might remember from the video, we have my great grandfather Worth Tucker who owned property in Elmo, Emery, Utah. The source of this information was a book. From the book we have 4 images: three that go with the source entry and one for a specific page that should be associated with the citation or source detail. Lastly, there is an extract from a page added as the citation text.

Here is what part of that looks like:

Family Tree Maker 2009

0 @I00001@ INDI
	1 NAME Worth /Tucker/
	1 SEX U

Legacy 7

0 @I1@ INDI
	1 NAME Worth /Tucker/
		2 GIVN Worth
		2 SURN Tucker
	1 SEX U

RootsMagic 4

0 @I1@ INDI
	1 NAME Worth /Tucker/
		2 GIVN Worth
		2 SURN Tucker

These next lines start an individual (INDI) record (notice the 0 prefix). The ID between @ characters (ex: @I00001@) uniquely identifies this person record from any other person records in the file. The name of the person is indicated in one or two ways. The NAME tag requires the full name with the surname between “/” characters. The name can also be broken in given name and surname as represented by GIVN and SURN. Some formats set the gender or sex to U for unknown or don’t include the tag unless it is set.

Included in the individual record is the property ownership event:

Family Tree Maker 2009

1 EVEN 80 acres
	2 TYPE Property
	2 DATE 1908
	2 PLAC Elmo, Emery, Utah

Legacy 7

1 EVEN 80 acres
	2 TYPE Property
	2 DATE 1908
	2 PLAC Elmo, Emery, Utah

RootsMagic 4

1 PROP 80 acres
	2 DATE 1908
	2 PLAC Elmo, Emery, Utah

FTM and Legacy choose to use generic event tag with a corresponding qualifying type of “Property” whereas RM simplifies it by using the property tag. Both are equivalent. Each includes the property description of “80 acres” and the date and place of ownership.

GEDCOM supports the concept of a master source with its information and multiple citation details corresponding to an event. In part 1, this concept was shown in the application screens in two screens (FTM 2009 and Legacy 7) or a single screen with different colored top and bottom sections (RootsMagic 4). The way this is represented in the GEDCOM format is to have a separate source record with a unique id and to reference that source record for the specific event being cited. Additional citation details are then given. Notice that the below section starts at level 2 and appears in the file at the same level as PLAC and right below it. This signifies that the event being cited is the parent level 1 or the property ownership event.

Family Tree Maker 2009

2 SOUR @S00002@
	3 PAGE 179
	3 DATA
		4 TEXT In 1908 Eliza Oviatt filed on eighty acres and Worth Tucker purchased
			5 CONC eighty acres of an adjacent school section. These properties became
			5 CONC the Elmo townsite, platted into lots that were sold to prospective
			5 CONC residents for $10.

Legacy 7

2 SOUR @S4@
	3 PAGE 179.
	3 DATA
		4 TEXT In 1908 Eliza Oviatt filed on eighty acres and Worth Tucke
			5 CONC r purchased eighty acres of an adjacent school section. The
			5 CONC se properties became the Elmo townsite, platted into lots t
			5 CONC hat were sold to prospective residents for $10.
	3 OBJE
		4 FORM jpg
		4 FILE C:\Users\mtucker\Documents\RootsMagic downloads - test\historyofemerycounty-page179.jpg
		4 _SCBK Y
		4 _PRIM Y
		4 _TYPE PHOTO

RootsMagic 4

2 SOUR @S1@
	3 PAGE 179
	3 DATA
		4 TEXT In 1908 Eliza Oviatt filed on eighty acres and Worth Tucker purchased e
			5 CONC ighty acres of an adjacent school section. These properties became the E
			5 CONC lmo townsite, platted into lots that were sold to prospective residents f
			5 CONC or $10.
	3 OBJE
		4 FORM jpg
		4 FILE C:\Users\mtucker\Documents\RootsMagic downloads - test\historyofemerycounty-page179.jpg
		4 _SCBK Y
		4 _PRIM Y
		4 _TYPE PHOTO
	3 _TMPLT
		4 FIELD
			5 NAME Page
			5 VALUE 179

The source tag (SOUR) here references the source record with the unique id between the “@” characters. We will look at its details in the next section. The level 3s and higher are for the citation detail. This citation is from page 179 of the source and text from that page is represented by the TEXT tag under the DATA tag. The concatenation tag is used to break up lengthy text onto multiple lines. Both Legacy 7 and RootsMagic 4 export the directory path to the image file of page 179. This is done with the object (OBJE) tag which specifies that the format of the file is jpg. The next three tags begin with an underscore character “_” which means that they are custom extensions to GEDCOM made by applications and that other applications are not required to support them. It appears that both Legacy and RootsMagic have chosen to support these tags. The _SCBK tag indicates if this image should appear in the scrapbook or media viewer inside the application with a Y value signifying “yes”. The _PRIM tag indicates if this is the primary image which is shown in the application in situations where only one image about the citation is shown. The _TYPE tag identifies this as being a photograph as opposed to some other media type. RootsMagic 4 has done something above and beyond the other vendors. It supports a custom template tag (_TMPLT) which we will investigate in a minute.

Before we continue it might be helpful to review page 646 from Evidence Explained to see how the Book: Basic format is represented as a source list entry, full reference note, and short reference note:

Take just a minute to examine the fields for each type, formating, as well as the ordering and contents of the author field.

The final section of the GEDCOM file that we will examine is the actual source record:
Family Tree Maker 2009

0 @S00002@ SOUR
	1 TITL Geary, Edward A., A History of Emery County
	1 NOTE
		2 CONC Geary, Edward A..  A History of Emery County:  .  Salt Lake City:
		2 CONC Utah State Historical Society, 1996.

Legacy 7

0 @S4@ SOUR
	1 ABBR History of Emery County
	1 TITL A History of Emery County
	1 AUTH Edward A. Geary
	1 PUBL Salt Lake City: Utah State Historical Society, 1996.
	1 OBJE
		2 FORM jpg
		2 FILE C:\Users\mtucker\Documents\RootsMagic downloads - test\historyofemerycounty-frontcover.jpg
		2 _SCBK Y
		2 _PRIM Y
		2 _TYPE PHOTO
	1 OBJE
		2 FORM jpg
		2 FILE C:\Users\mtucker\Documents\RootsMagic downloads - test\historyofemerycounty-titlepage.jpg
		2 _SCBK Y
		2 _TYPE PHOTO
	1 OBJE
		2 FORM jpg
		2 FILE C:\Users\mtucker\Documents\RootsMagic downloads - test\historyofemerycounty-copyrightpage.jpg
		2 _SCBK Y
		2 _TYPE PHOTO

RootsMagic 4

0 @S1@ SOUR
	1 ABBR History of Emery County
	1 TITL Edward A. Geary, A History of Emery County (Salt Lake City: Utah S
		2 CONC tate Historical Society, 1996), [Page].
	1 _SUBQ Edward A. Geary, A History of Emery County, [Page].
	1 _BIBL Edward A. Geary. A History of Emery County. Salt Lake City: Utah S
		2 CONC tate Historical Society, 1996.
	1 OBJE
		2 FORM jpg
		2 FILE C:\Users\mtucker\Documents\RootsMagic downloads - test\historyofemerycounty-frontcover.jpg
		2 _SCBK Y
		2 _PRIM Y
		2 _TYPE PHOTO
	1 OBJE
		2 FORM jpg
		2 FILE C:\Users\mtucker\Documents\RootsMagic downloads - test\historyofemerycounty-titlepage.jpg
		2 _SCBK Y
		2 _PRIM N
		2 _TYPE PHOTO
	1 OBJE
		2 FORM jpg
		2 FILE C:\Users\mtucker\Documents\RootsMagic downloads - test\historyofemerycounty-copyrightpage.jpg
		2 _SCBK Y
		2 _PRIM N
		2 _TYPE PHOTO
	1 _TMPLT
		2 TID 372
		2 FIELD
			3 NAME Author
			3 VALUE Edward A. Geary
		2 FIELD
			3 NAME Title
			3 VALUE A History of Emery County
		2 FIELD
			3 NAME SubTitle
		2 FIELD
			3 NAME PubPlace
			3 VALUE Salt Lake City
		2 FIELD
			3 NAME Publisher
			3 VALUE Utah State Historical Society
		2 FIELD
			3 NAME PubDate
			3 VALUE 1996

FTM takes a shortcut and using the format specified for source list entry, puts that as the source note. The title (TITL) comes from how FTM automatically named the source when it was created. There are two things to note about the NOTE text: 1) it does not have a way to indicate that the book title should be italicized and 2) there appears to be extra periods and spaces in it.

Legacy 7 takes the approach of trying to stuff the EE-style citation into the fewer fields available for the “old-style” citation. There are 6 parts of a source list entry citation for a basic format book:

1. Author
2. Main Title
3. Sub Title
4. Place of publication
5. Publisher
6. Year

Legacy 7 implements the basic book template with the following fields (the last two pertaining to the citation detail):

1. Author Last Name
2. Author Given Name(s)
3. Author Suffix
4. Title
5. Short Title
6. Publisher City
7. Publisher State
8. Publisher
9. Publish Date
10. Volume Data
11. Page
12. Volume

The standard fields available in GEDCOM are:

1. Title
2. Author
3. Publication

So Legacy makes the following matches:

• GEDCOM Title = Title
• GEDCOM Author = Author Given Name(s) + Author Last Name + Author
• GEDCOM Publication = Publisher City + Publisher State + Publisher + Publish Date

Note that there is some formatting according to the source list entry and can be seen in the PUBL tag as it follows the order of fields and contains the colon and comma in the correct location. Also, the abbreviation (ABBR) tag is used to name the source in the master list after it is imported. The rest of the GEDCOM contents from the Legacy 7 file specify the 3 media files associated with the source. Nothing new there.

I have yet to do additional experiments to determine how the translation to “old style” citations works with more complicated citation formats.

Finally we look at RootsMagic 4. It also uses the abbreviation and object tags in the same way as Legacy 7. But some interesting things are happening in the rest of the file. Notice that the title tag follows the format for a full reference citation complete with parenthesis, commas, and colons. The title is between special formatting tags < i > and < /i > to indicate that it should be italicized. Where the page number should go is the textual placeholder “[Page]”. The custom subsequent tag (_SUBQ) contains the short note format although it should just contain the author’s last name. The custom bibliography tag (_BIBL) contains the source list entry format. It appears that a bug in the export is causing the bibliography entry to not show author with last name first. It is important to note that any application that imports a RM4-generated GEDCOM will get only the contents of the title tag and will have to manually edit it to remove the italicization indicators which they don’t support.

Now let’s get to the part where RootsMagic 4 has shown some innovation in their GEDCOM. Remember the custom template (_TMPLT) tag we saw for the citation:

3 _TMPLT
	4 FIELD
		5 NAME Page
		5 VALUE 179

There is also one in the source:

1 _TMPLT
	2 TID 372
	2 FIELD
		3 NAME Author
		3 VALUE Edward A. Geary
	2 FIELD
		3 NAME Title
		3 VALUE A History of Emery County
	2 FIELD
		3 NAME SubTitle
	2 FIELD
		3 NAME PubPlace
		3 VALUE Salt Lake City
	2 FIELD
		3 NAME Publisher
		3 VALUE Utah State Historical Society
	2 FIELD
		3 NAME PubDate
		3 VALUE 1996

Now compare that with the source entry screen:

Notice that in the yellow Master Source section, there are 6 entry fields: Author, Title, Sub-title, Publish Place, Publisher, and Publish Date. These correspond to the 6 template (_TMPLT) field name entries in the GEDCOM: Author, Title, SubTitle, PubPlace, Publisher, and PubDate. In the green Source Details section Page corresponds with the field name entry is the citation section: Page. The value tags contain the actual value. That way the details of knowing individual fields and values is not lost. Completing this is the template id or TID tag that is a unique number used internally by RootsMagic 4 to always refer to this template. That is why you can never edit existing templates in RootsMagic 4.

Here are the details of the template for id 372 as shown in the Source Templates screen:

It is interesting that each field is given a type to indicate if it is a Name, Place, Date, or Text. This could come in handy in future situations. Imagine searching all sources not just for the text “White” but for all sources that contain a name that contains “White.” Searches like that would return more appropriate results.

So what have we discovered is that the current three applications that support EE-style templates do so slightly differently on the input side (part 1) and vary greatly when it comes to GEDCOM output. As it stands today much is lost in the GEDCOM export rendering rich citations into blobs of text. RootsMagic 4 solves this problem in a proprietary way using its own template id and template fields names.  Currently no real interoperability exists between these applications when it comes to EE-style source citations.
This post is already long enough and I will likely expound on my ideas in a follow-up post. But imagine the RM4 implementation standardized and universally accepted. What a world of interoperability that would open up!

There is so much to think about. What do you think?

She explains the issues clearly and interviews genealogy blogger, Stephen Danko to get his opinion. Lisa also gives the outcome of her interview requests with Ancestry and World Vital Records.

I was excited to hear the interview with Stephen as I have been an admirer of his work for years ever since the Genealogy Guys first mentioned him on their podcast.  Stephen’s genealogy blog is in actuality an online research log where he posts document images, transcriptions and translations from his research. Like all genealogists should do, he cites all sources following Evidence Explained. In fact, I had his website in mind when I created the sample site used in the video. For many months, whenever I visited his blog I would imagine a Download link next to each of his source citations. Stephen is somebody I would love to meet. Maybe NGS 2010 in SLC?

Lisa, thanks for getting the word out.  This is truly a grassroots effort and I cannot do it on my own.  Keep spreading the word and contact the providers of the software and services you use.

Thank you!

For those who want to know more, here are the details.

We will first start with the QuickCheck models found in Evidence Explained. These models can be used by software developers as a feature specification:

This example from page 646 specifies that a basic book citation consists of 7 parts:

1. Author
2. Main Title
3. Sub Title
4. Place of publication
5. Publisher
6. Year
7. Page

It also indicates the format of the citation specifying where to put commas, colons, periods, and parenthesis as well as when text is italicized.

These QuickCheck models as well as other citation formats specified in EE (Evidence Explained) have been coded into Legacy 7, Family Tree Maker 2009, and RootsMagic 4.

Now even though each of these 3 desktop genealogy applications used EE for their spec and received clarification from their outside business analyst, Elizabeth Shown Mills, there are slight variations in the implementation by each. To prove this, here are comparisons of each application and how it deals with a book citation:

RootsMagic 4

Legacy Family Tree 7

Family Tree Maker 2009

Here is a comparison of these three applications showing different interpretations of the EE citation model:

So currently we have 3 applications that support EE and each are slightly different. What is needed is a standard that each can measure up against so as more applications support these citation formats, tests can be created to verify compatibility. Some sort of consortium needs to be created to discuss the current differences and come to a consensus of opinion.

Let’s say that process has already occurred and agreement has been reached. Now a standardized file format can be designed to handle the additional level of detail required for the citation models. In some ways this file format would serve a similar purpose as GEDCOM, but would be updated with the capability of handling more-detailed source citation and referencing media such as images and files.

When visiting a website that supports this new file format, the researcher will encounter a download button or link that references that file:

In the above example, the link references a file called book.cite. The .cite extension represents a file of a specific content type (called a MIME type) identified as: application/cite+xml. The important thing to know about this is that a .cite file can now be uniquely identified from other content types.

We will not discuss the actual structure of the file at this point. Details will be provided in a later post.

When the researcher clicks the link, the web browser (in this case Firefox) prompts us if we would like to save or open the file:

As can be seen from the screen, the file is correctly identified as a CITE file and picks the default application used to open the file, ClickCite Launcher.
Part of the prototype code that I wrote was an application called ClickCite Launcher. Its purpose is to intercept CITE files and pass the file along to an importer application. The ClickCite Launcher application would need to be installed on the researcher’s computer and part of the installation would make a file association between .cite files and ClickCite Launcher. This is how Firefox knows which application to use.

The launcher application and the file association are for a computer running Windows. I am not familiar with Mac computers in this regard, but it would surprise me if a similar capability was not available.

This launcher application is aware of all installed applications that support importing of CITE files and presents the user with a list of desktop genealogy applications:

After an application is picked from the list and Import is clicked, the launcher application would start the importer application and pass it the book.cite file. The launcher application would likely be open source software whereas each desktop application that supported CITE files would provide their own importer. In the video demonstration, the importer for RootsMagic 4 presented no user interface and just updated the database file for the most-recently opened database, test.rmgc. The developers of the importer could choose to show a user interface that might include a list of all previously created databases and allow the user to choose which database would be the target of the import.

The process is depicted in the following diagram:

1. Genealogist clicks download link which causes the browser to download the file onto his/her computer.
2. When the file is manually opened or opened by the browser when the Open option is selected, Windows runs the application associated with the extension. In this case, .cite is associated to the launcher application. The launcher receives the location of the downloaded CITE file.
3. When Import is clicked on the launcher, the importer for the selected genealogy software is started and passed the location of the downloaded CITE file.
4. The importer loads the CITE file and adds information to the application’s database file.

I hope this addresses some of the technical questions that I have been receiving.

Thank you for contributing to the conversation. Any discussion of sources and their classification as original or derivative is not complete without discussing source provenance. When we trace the incarnations of a source all the way back to the original, we are able to do two things: 1) answer the question “Is there a better source?” and 2) determine independent origin.
Let us take the example of an original census that was microfilmed and then digitized. We are looking at the digital copy and we determine that the image is dark and hard to read in certain areas. We ask ourselves if there is a better source and determine that the problem was likely with the microfilm so if a better source exists it would have to be the original. Let’s say that we are able to consult the original and we can read the problem areas. In this case, the original was the better source.

Let’s change our scenario a little. Same original and two derivatives, but this time we ask to consult the original and it doesn’t exist as it was destroyed after microfilming. In this case the microfilm copy or the digital image is the best source available. Similarly, if the original exists but has deteriorated in the years since it was microfilmed, then the microfilm/digital image would be the better source.

When we have an original, we only have two ways to make a copy of it: 1) photographically reproduce it or 2) manual copy (either written or typed).

To handle the first group, Evidence Explained discusses the concept of an Image Copy. An Image Copy is a derivative, but it can be treated as if it were the original if it 1) is legible and 2) doesn’t conflict with other evidence. A #3 is implied in that all representations of the source back to the original must also be able to be treated as originals. So the microfilm is an Image Copy of the original census and can be treated as an original. The digitized image of the microfilm can also be treated as an original because it is an Image Copy and if we print out that digital image we have made another Image Copy which as long as it passes the tests can be treated as an original. In your example, the death certificate is an original source and a photocopy of it is an Image Copy of an original and can likely be treated as if it were an original.

For the manual copy example, we have more choices: Duplicate Original, Record Copy, Transcript, Extract, and Abstract. The census enumerator goes house to house and records the information that is required. It is the original source. Now the Federal government requires that a copy be sent to them, so the enumerator makes a copy of the original. Technically it is a well-known type of derivative called a transcript, but because it is part of the “official” act of creating the census, then we can treat it as if it was the original and we give it the special name of Duplicate Original. There are known examples where this copy was changed to be alphabetical or only listed people by their initials or mistakes were made because the copying was done column by column instead of row by row, etc. There are even case where the true original was sent to the Federal government and the Duplicate Original stayed at the state level. We try not to get too caught up in the details and are satisfied that it is a Duplicate Original all the while keeping an eye out for irregularities. Other examples of Duplicate Originals are grantor/grantee copies of a deed and counterparts. I think about copies made as part of the same “transaction” when I think of Duplicate Originals.

A Record Copy is also technically a derivative (likely a transcript) but this time an official (likely a clerk) is entering information into a register. As in other cases, mistakes can still be made but because of the authority of the official and their mandate to record the information, we can consider both the original deed and its recording in the deed book as originals. There have also been times when the transaction has been recorded directly into the register in which case it is the original.

Outside of the special cases of Duplicate Original and Record Copy, other manual copying is categorized as transcript, extract, and abstract based on the amount of document that is copied. Transcript is full document (the entire census for a year and state), extract ( a page from the census), and an abstract (just the more important parts: names, places, dates, etc.)

All the above helps us determine the best sources for our analysis.

Let’s briefly talk about independent origin.  Let’s say we have 100 derivative sources that are varying generations removed from their original.  Two sources agree on one birth year (1880 ) whereas the other 98 give the birth year as (1830).  We might be tempted to go with the 1830 year as there exist more sources that agree with that year.  But without tracing the provenance, that would be a mistake.  Let say that the provenance is traced and all 100 derivatives come from only 3 independent original sources.  In fact, all 98 derivatives that agree with 1830 are all from the same original source.  When we consult this original source, we see that a mistake was made in the transcription that changed it from 1880 to 1830.  The concept of independent origin is important to consider.  Evidence Explained reminds us that quantity can never trump quality.

As genealogists, we identify source, the information that they contain, and the kind of evidence that information provides when compared to our research problem. Sources, information, and evidence. Not one is more important than the other. That would be like saying that our eyes or hands or mouth is more important than the others.

Earlier today I posted the following to the APG list on RootsWeb:

There exists confusion in the current genealogy literature on the definition of an original source.

For this discussion I would like to focus only on the definition of original source and not derivatives, common derivatives (transcript, extract, abstract), or derivatives that can be treated as originals (image copy, record copy, or duplicate originals). I want to focus on the source – the container, the person, the paper, the stone, the object. Not the information contained in it (as much as possible) and its classification as primary or secondary. Also I don’t want to focus on how that information relates to the research question (i.e. the evidence and whether it is direct, indirect, or negative).

The 4 main sources that genealogists can turn to for a definition of original source are: Evidence! (1997), The BCG Standards Manual (2000), Professional Genealogy (2001), and Evidence Explained (2007). But using these sources can be contradictory and confusing. Is this due to the refinement of the definition over the years?

Let’s look at some specifics.

Evidence!

On page 49 of Evidence! the word material is used instead of sources, but the definition is useful:
“Original material, as defined by the purist, is based on firsthand knowledge – be it oral or written.”
The main criteria identified is: firsthand knowledge

The BCG Standards Manual

As the GPS was formalized, The BCG Standards Manual defined original sources (see page 8 ) as:
“the person or record whose information did not come from data already spoken or written.”
The main criteria: first occurrence of information

Professional Genealogy

The next year, Professional Genealogy was published which includes Chapter 17, “Evidence Analysis”. On page 333, we find the definition of an original source as:
“In genealogical terms, original sources are those that meet two criteria.
They are made at or near the time of the event, and their informants are in a position to know the facts firsthand.”
So a two-pronged test must be passed in order for a source to be original.
Criteria: timeliness and firsthand knowledge

Evidence Explained

In 2007, the definition changes to focus on form as found in Evidence Explained. The research process map inside the front cover has “form” written under both original and derivative. We find a definition of original source in three places:
. page 24: “Original sources – material in its first oral or recorded form”
. page 826: “original source: a source that is still in its first recorded or uttered form.”
. page 828: “source: … Sources are broadly classified as either an original source (q.v.) or a derivative source (q.v.), depending upon their physical form.”
The main criteria: physical form

So it appears that now a single-question test would be sufficient to classify a source as original:
“From what was this source derived?”
If the answer doesn’t reveal another source, then it is an original.

In an APG discussion, Elizabeth Shown Mills indicates that “Original sources can have secondhand information.” An example is an original death certificate that contains primary information of the death but secondary information of the birth. A question that I haven’t seen answered is:
“Can an original source contain ONLY secondary information?”
What would be an example? If that case is true, then only the definition in Evidence Explained is useful. Otherwise you mix the classification of source with that for information.

Primary Source

Another point I want to bring out from Evidence Explained is the definition of primary source found on pages 22-23:
“PRIMARY SOURCE
. one created by someone with firsthand knowledge . one created at or about the time an event occurred Within this framework, contradictions abound between theory and practice, causing ambiguous analyses and unreliable conclusions.”

It appears that the reason primary source is discussed is to identify its weaknesses and show why original source is preferable in genealogy research.
The problem stems from the two-prong test of firsthand knowledge and timeliness. If a source came from someone with firsthand knowledge but was created long after the event in question, the test would fail. Or, if the source was create near the time of the event, but the informant had secondhand knowledge, then the test would also fail. The strength of the single test of physical form for an original source is its simplicity on focusing on the object that is the source and not the informant, his/her knowledge, or the information.

What is confusing is that what Evidence Explained defines as a primary source with its problems is the same definition that Professional Genealogy uses for original source.

My inclination is that this progression of definitions is a refinement of understanding over the years and that the current accepted definition of original source focuses only on physical form. And as the other three guides are revised, they will share the definition currently found only in Evidence Explained.

The Back Story

Now let me tell you the back story that produced this examination of the definition of original source. In a recent presentation aimed at teaching beginning researchers the Genealogical Proof Standard, the question came up as to whether a grave marker is an original or derivative source. In preparation for the presentation (and as part of a study assignment for the ProGen Study Group) I shared the two-pronged test defined in Professional Genealogy, Chapter 17. Because it was a double marker and the wife died three decades after the husband, the class questioned the timeliness part of the test. I sensed that there was still a little confusion on this point and have wrestled with this question since then. There has to be a way that beginners can confidently determine if a source is original or not.

As it so happens, this month’s ProGen Study Group assignment includes reading the first chapter of Evidence Explained. That is when I noticed the possible shift toward physical form and a single test. I had read the chapter a few times before, but now I saw it differently. Now a single test, that is something that I think beginners can understand!

So in the grave marker case, what would it be derived from? One answer could be nothing, in which case it is an original. In some cases a new grave marker is created to replace an old one. I don’t feel that happened in this case. It would be something to check out. When my mother passed away, we worked with the mortuary to design the grave marker and then months later it was created. We verified the paperwork and later the grave marker to make sure all information was as we expected. So, was the marker inscription a granite-carved extract from the original paperwork or more likely a duplicate original? In either case, things are less problematic and I feel comfortable classifying the grave marker as an original source.

Classifying Sources with Evidence Explained?

Something else I noticed in my latest browsing of Evidence Explained.
Chapter 5 deals with Cemetery Records and on page 207 is summarized the QuickCheck models which are grouped in three categories: Cemetery Office Records, Markers & Memorials (Originals), and Derivatives. So it appears in this case I don’t have to worry about things too much and can call the grave marker an original source because Evidence Explained classifies it as such.
I think that for beginners using Evidence Explained as a crutch is acceptable as they gain confidence is classifying sources using the physical form test. So my recommendation for beginners would be to 1) try to determine if a source is original or derivative on their own and come up with an answer, 2) look up the source in Evidence Explained and if original or derivative is specified compare it with their answer. If they don’t match, try to determine why it might be the other classification.

Other chapters that appear to at least partially classify sources as original or derivative (or image copy) are: Ch 6 – Census Records, Ch 7 – Church Records, Ch 8 – Local & State Records: Courts & Governance, Ch 10 – Local & State Records: Property & Probates, Ch 11- National Government Records, and Ch 12 – Publications (Books, CDs, Maps, Leaflets & Videos).

Too bad there is not a growing master list of record types with classifications as: original, derivative, image copy, duplicate original, and record copy.

Now, two hours later, I have down in words what has been swarming in my head for weeks. I look forward to learning from your understanding and experience.

Update:

Follow the discussion on the APG list.

A discussion about modern American genealogy cannot begin without first recognizing Donald Lines Jacobus.

“During his lifetime, Jacobus was widely regarded as the dean of American genealogists, and he is recognized as the founder of the modern school of genealogy in the United States.  On his death, he was described by his colleague Milton Rubincam, as ‘the man who more than any other single individual elevated genealogy to the high degree of scholarship it now occupies.’”  See National Genealogy Hall of Fame.

His many works include Genealogy As Pastime and Profession which was published in 1930.

In 1940, The American Society of Genealogists (ASG) was founded to “elevate the profession of genealogy to the same literary and scientific level enjoyed by history.”

In 1964, The Board for Certification of Genealogists (BCG) was founded by a few fellows from ASG, members of NGS and others.  The purpose of BCG was to “set scholarship standards for professional genealogists.”

Noel C. Stevenson was a lawyer and genealogist who tried to bring principles of evidence from the field of law to genealogy.  He proposed that genealogists follow the principles of Preponderance of the Evidence, a standard used in civil cases.  For the next two decades, genealogists used the POE although with a higher standard than used in law.

One of Stevenson’s important works was Genealogical Evidence: A Guide to the Standard of Proof Relating to Pedigrees, Ancestry, Heirship and Family History which was published in 1979.

Also in 1979, Gary B. and Elizabeth Shown Mills wrote an article for The Genealogical Helper titled, “How to Properly Document Your Research Notes.”  This helped spawn the idea of writing a citation guide specifically for genealogists.  Richard S. Lackey took up the project and published Cite Your Sources in 1980.

In 1997, the BCG dropped the use of POE and adopted the Genealogical Proof Standard. In that same year, Elizabeth Shown Mills published an updated citation guide, Evidence! Citation & Analysis for the Family Historian.  In 2000, the BCG published the GPS in the BCG Standards Manual.

In 2001, Professional Genealogy: A Manual for Researchers, Writers, Editors, Lecturers, and Librarians was published with Elizabeth Shown Mills as editor.

In 2007, Elizabeth Shown Mills’ work, Evidence Explained: Citing History Sources from Artifacts to Cyberspace, is published – almost 30 years from that first citation article published in The Genealogical Helper.  The first article was 5 pages whereas Evidence Explained contains 885 pages.

I am open to any other suggestions of what to include on this timeline.

.

Previous versions of Legacy allowed for source citations, but not anywhere near this level.  So this improved citaion feature can be considered an incremental innovation.  Evidence Explained (or EE ) is 885 pages and contains around a thousand citation models for U.S. and international documents.  Just reading the book is an accomplishment in itself but then translating that into software? Amazing!

EE was published around August 2007 and Legacy 7 was released in June 2008.  I know that Geoff Rasmussen put in many hours during those months reading and re-reading EE, thinking about it, talking with ESM, translating citation models into software requirements, creating templates, and then testing them. I dare say that Geoff is in an elite class of a few that know this book backwards and forwards. Ken McGinnis and Dave Berdan spent many hours coding SourceWriter and the other Legacy 7 features.

With SourceWriter, a genealogist can more easily find the appropriate template among so many and then fill in the blanks with the needed information.  There is no question of which citation information is needed for the document that you are citing.

Quite possibly because of the innovative work of the Legacy team, other genealogy software applications now support or will soon support EE source templates.  If my 2009 genealogy predictions come true, more and more genealogy applications (desktop and online) will support these templates.  They will become a new standard.  So does that mean that this feature is an incremental change with side effects or a transformational change?

Let me end with a story.  For many years I have thought about EE’s predecessor, Evidence!, and how it could be implemented in software.  When EE  was published, I got more excited as I saw how the QuickCheck Models could be translated into software requirements.  It would take a lot of effort and time, but the time felt right.  I would catch myself thinking about this over and over, doing preliminary designs in my head.  One day I was driving home from work listening to a DearMYRTLE podcast interview with Geoff.  He was leaking some information about Legacy 7.  When he said something to the effect of “source citation models following Elizabeth Shown Mills’ Evidence Explained” I literally cried out “Nooooo!”  They had beat me to it.

I have since met the Legacy team and consider them as friends.  This is the first time they have heard this story.  They have had no warning about this post and will likely be surprised that they are the winners of the second ThinkGenealogy Innovator award.

As 2008 closes, we stop to ponder what awaits genealogy in 2009.  In coming up with this list, I have no insider information.  I simply looked at the information publically available and tried to determine what is possible or likely for the upcoming year. 

So here is my list of 9 genealogy predictions for 2009:

1. Two more desktop genealogy applications will support source citation templates from Elizabeth Shown Mills’ Evidence Explained. Currently Legacy 7 and RootsMagic 4 support this. The other two might likely be Family Tree Maker and The Master Genealogist.
2. One major online database (Ancestry, WorldVitalRecords, FamilySearch, Footnote) will announce upcoming support for Evidence Explained source citations.  Other sites will soon follow with their own announcements.
3. FamilySearch Research Wiki will grow to over 30,000 content pages (excluding talk and stub pages) by the end of the year. As of 28 December 2008, there were 12,573 content pages.
4. FamilySearch will announce a replacement for GEDCOM. GEDCOM will still be supported for many years as software will support it as well as the newer format. Reasons for the new format will be to better support source citations and media.
5. I hope I am wrong about this one, but with the current economic downturn we could see financial pressure force one of the more-recently-started genealogy companies to merge with or be acquired by another company.  Worse case scenario, it could go bankrupt.
6. Many more genealogy societies will join the social networking site, Facebook, by adding a group page. Searching groups today for “genealogical society | genealogy society” (the pipe symbol “|” means “OR”) returns 80 results. I wouldn’t be surprised if by the end of 2009 the total was over 200.
7. More genealogy applications will be available for the iPod Touch and the iPhone. Currently the iTunes App Store only lists three applications: MobileFamilyTree, FamViewer, and Shrubs.  This last holiday season, the iPod Touch was a hot item.  I wonder how many genealogists own one now.
8. Through blogs and podcasts, genealogists will hear more about the benefits of Web 3.0 (also called the Semantic Web) and how it applies to genealogy.
9. Genealogists are puzzle solvers when it comes to researching their family history, but do they play games?  I expect to see a genealogy-related game released in 2009.  It could be a casual game for Facebook or something more interactive for the Wii.  How about a Mii Familii Trii?  The game might teach research-related concepts (citation, transcription, etc) or (via GEDCOM import) put your ancestors in the game. 

How about you?  What do you think genealogy will be like in 2009?  Don’t be shy about posting a comment.

This time next year we will revisit this list and see how well the predictions compare with reality.

Ten years passed between the publication of Evidence Explained and its predecessor, Evidence! Citation & Analysis for the Family Historian.  Even with the passing of a decade, I consider Evidence Explained an incremental innovation that has caused some beneficial side effects. 

Incremental Innovation

Whereas Evidence! simply gave citation examples for primary, subsequent, and bibliographic entries, Evidence Explained gives citation examples, explanation of record types, and QuickCheck Models:

Each QuickCheck Model shows a sample citation that is annotated with descriptions of what type of information goes where in the citation.  The model shown above is for a digital image of a census record on an online commercial web site (see EE, page 240).  The parts of a citation are identified as Census ID, Jurisdiction, Schedule, Civil Division, Page ID, Household ID, etc.  The model looks like a form that was then filled in with a citation.  The innovation of the QuickCheck Model for citations makes it easier to understand the citation format and apply it to your own sources.

Side Effects

A side effect of this innovation is that now citation models following Evidence Explained are showing up in genealogy software.  The format of the QuickCheck Model also makes it easy for developers to code the templates into their software.  Two examples are Legacy 7 and RootsMagic 4.  It is likely that this trend will continue as it has already been announced as an upcoming feature of Family Tree Maker 2009.  As more and more software applications include citation models from Evidence Explained, it will become a standard and one day be as commonplace as GEDCOM is today. 

Watch out for online database sites such as Ancestry, FamilySearch, Footnote, and WorldVitalRecords to see which is the first to follow Evidence Explained.  Whether or not QuickCheck Models were created with software developers in mind, it is apparent that this incremental innovation is having some real side effects.

To learn how to make a ThinkGenealogy Innovator Award nomination, check out the Innovator Awards page.

I like the fact the WorldVitalRecords geocodes all records added to their site.  Why you are at it, why don’t you add source citations in metadata/xml form following the conventions in Elizabeth Shown Mills book, Evidence Explained: Citing History Sources from Artifacts to Cyberspace? 

Currently source citation is hard.  When it is available, it is in text format that must be copied and pasted into your genealogy program.  But source citation is vital so that proper evaluation of evidence can be done and so that constant re-examination of the same records can be avoided.  If when you click on a document to download the image, the link was instead something like an rss link that has metadata with it (think rss enclosure tag) and if that xml format were a standard then genealogy software could read the information, add the image to the application, and add the proper source citation.  What could be easier for a user than every time a document image is downloaded from an online database, the source was automatically cited?  The software developers would be half way there as they would then just need to add a way to manually add the same information for offline sources. 

The first analysis that needs to be done with a source is to determine if it is original or derivative.  The metadata could include this information already.  The next step would be to have the metadata for derivative sources include the source provenance all the way back to the original.  Who would be in a better position to know that than the site owner who negotiated with the owner of the source content?  This identification would then only have to be done once correctly and it would save many family historians/genealogists from doing the same work and sometimes incorrectly. 

Now the metadata would also be available to search engines and special source searches could be created to find and aggregate the information.  Think about what Google, Technorati, Digg, del.icio.us, Facebook or others could do with this type of information.

1. Creating a source citation metadata standard. 
2. Being the first records site to metadata source cite all their content. 
3. Making it extremely easy to cite online sources. 
4. Creating a whole new way to search for records. 

Now talk about jumping curves!

Some of these ideas I have shared before in Expanded Vision of Genealogy 2.0.

Happy curve jumping.

Web 2.0

The term Web 2.0 has been around since 2004 and is defined by wikipedia as the:

“perceived second generation of web-based communities and hosted services — such as social-networking sites, wikis and folksonomies — which aim to facilitate collaboration and sharing between users”

There is much debate over the definition of Web 2.0 and what makes a website “Web 2.0″.  According to SEOmoz.org, some of the defining characteristics of Web 2.0 are:

• User generated and/or user influenced content
• Applications that use the Web (versus the desktop) as a platform, in innovative ways
• Similar visual design and shared functional languages
• Leveraging of popular trends, including blogging, social tagging, wikis, and peer-to-peer sharing
• Inclusion of emerging web technologies like RSS, AJAX, APIs (and accompanying mashups), Ruby on Rails and others
• Open source or sharable/editable frameworks in the form of user-oriented “create your own” APIs

Web 2.0 links:

• O’Reilly Radar - Web 2.0 Compact Definition: Trying Again
• SEOmoz’s Web 2.0 Awards
• WebTrends 2.0 - Web 2.0 Directory
• Pixel Acres - The Visual Design of Web 2.0
• Web Design from Scratch - Current Style in Web Design

Sample Sites:

• Flickr
• del.icio.us
• LinkedIn
• Wikipedia

Genealogy 2.0

When I search the internet for “genealogy 2.0″, I get a number of sites that talk about the application of Web 2.0 to genealogy.  These sites mention social networking and collaboration as key components of Genealogy 2.0.  One blog, The Plog: Pytlewski Log, states:

“traditionally genealogy 2.0 has only referred to the new internet based applications that are changing the way we collaborate as a genealogical community”

My view of Genealogy 2.0 is broader than Web 2.0 genealogy application or what I term, Web 2.0+Gen.  Maybe it is because I have developed both web applications and Windows client applications.  Maybe it is because I see so many areas for improvement and innovation in genealogy software and I don’t want to wait around for Genealogy 2.5 or 3.0.  Or maybe it is just the developer in me that wants to avoid tight coupling. But pairing Genealogy 2.0 with Web 2.0 excludes genealogy software that is not web-based.  It also seems to focus too much on what Web 2.0 is and not what Genealogy 2.0 could be.

Genealogy 2.0 links:

• Genealogy 2.0
• Family Matters

Sample Sites:

• FamilyLink
• Geni
• FamilyPursuit

Expanded View of Genealogy 2.0

Many of these ideas are not new, but have been in the genealogy community for years.  The time is ripe for them to materialize as software that will aid genealogists and family historians to do things that they have never been able to easily do before. 

An expanded view of Genealogy 2.0 includes the following characteristics:

• Social networking 
• Collaboration during research, analysis, and conclusions
• More than just sharing data and results
• Supports sources, information, evidence, and conclusions
• Document-centered data collection
• Standardized source citation (see Evidence Explained)
• Source citation as data not text
• Source provenance
• Information extraction
• Evidence evaluation and weight
• Conclusion recording
• Online data backup
• Community of researchers
• Online data storage or peer-to-peer offline storage
• Data linking and layering, not merging
• Expanded to include not only web-based applications but also desktop and mobile
• Modernizing of GEDCOM or replacement with XML-based format
• The ability to not do anything with genealogy for a year and then start right where I left off without any loss of information or momentum

Now the last point may just be my own personal wish list item, but if  a Genealogy 2.0 application included a place to put everthing and kept track of what I have done and what else needs to be done then it would be much easier to continue where I left off.

Genealogy 2.0 Expanded links:

• Source-Centricity
• Evidence Explained: Citing History Sources from Artifacts to Cyberspace by Elizabeth Shown Mills
• The BCG Genealogical Standards Manual

I look forward to your comments and ideas about Genealogy 2.0.

The book can be purchased from Genealogical Publishing Company.