In Better Online Citations – Details Part 1 we examined how the QuickCheck model for “Book: Basic format” from Evidence Explained was coded in Family Tree Maker 2009, Legacy 7, and RootsMagic 4. From the screens we were able to identify implementation differences between the three applications. There are also differences between the applications in how citation information is conveyed via a GEDCOM export. The individual fields shown on the template screens are lost in the standard GEDCOM export making it impossible to create a rich EE-style citation in one application, export it to GEDCOM, and import it into another application while retaining that richness. In all cases (except when the exporter and importer of the GEDCOM is RootsMagic 4), the citation is changed from a “Book: Basic format” to a generic “old-style” (pre EE) format with important details lost.

In the previous post I skipped all details of the file format needed to support online downloadable source citations. From my own observations, from survey feedback as well as comments on this blog and through e-mail there are two main camps when it comes to this topic. One group feels that the best approach to take is an extension to the existing GEDCOM 5.5 standard (which was released in 1996). They feel that it is the best choice to lower the barrier of adoption and avoids YAFF (Yet Another File Format). See the comments from Tamura Jones from Part 1. The second group is open to a new file format based on XML (Extensible Markup Language) which has wide support among programming languages. As a programmer I lean slightly to the side of XML and that is what I used in the prototype shown in the video. But I am open to either view.

In that spirit of openness, I will first look at how the three applications that support EE-style citations represent those citations in an exported GEDCOM file. In this post, we will look at sections of GEDCOM which is a little technical but there will be plenty of explanations for readers of all levels. From the previous post we saw differences in how each application implemented the EE-style QuickCheck model and this post will show differences in how each exports them. You will also see some forethought in the export from one vendor.

For those who have never seen a GEDCOM file, each line starts with a number: 0, 1, 2, etc. A zero is the beginning of a new record and higher numbers are “nested” under lower numbers. It is a way of grouping information together. In the GEDCOM fragments I show here, I will indent the lines to make them easier to understand. I have also done some slight rearranging of lines to make the comparisons easier. After the number, each line has a tag which is a shortened identifier of the information contained on that line. You can see definitions of the standard tags here.

A GEDCOM file starts with a header that describes the software application that generated the file as well as the version of the GEDCOM standard:

Family Tree Maker 2009

0 HEAD
	1 SOUR FTM
		2 VERS Family Tree Maker (18.0.0.305)
		2 NAME Family Tree Maker for Windows
		2 CORP The Generations Network
	1 DEST GED55
	1 GEDC
		2 VERS 5.5
		2 FORM LINEAGE-LINKED

Legacy 7

0 HEAD
	1 SOUR Legacy
		2 VERS 7.0
		2 NAME Legacy (R)
		2 CORP Millennia Corp.
	1 DEST Gedcom55
	1 GEDC
		2 VERS 5.5
		2 FORM LINEAGE-LINKED

RootsMagic 4

0 HEAD
	1 SOUR RootsMagic
		2 VERS 4.0
		2 NAME RootsMagic
		2 CORP RootsMagic, Inc.
	1 DEST RootsMagic
	1 GEDC
		2 VERS 5.5.1
		2 FORM LINEAGE-LINKED

The first line starts the header record which contains a source program (SOUR) that generated the file and a destination program or format (DEST) that will use the file. The level under source shows the program, version and company information. The GEDC or GEDCOM tag identifies that the file adheres to the specified GEDCOM version and format. FTM and Legacy shows 5.5 whereas RM shows 5.5.1. GEDCOM 5.5.1 was published as a draft in 1999 but the changes between it and GEDCOM 5.5 do not affect our discussion. The rest of the header record was removed as it is not important for our comparison. Even though the numbering and abbreviations take a little getting used to, the file format is pretty straight forward. The rest of the file is just as understandable.

As you might remember from the video, we have my great grandfather Worth Tucker who owned property in Elmo, Emery, Utah. The source of this information was a book. From the book we have 4 images: three that go with the source entry and one for a specific page that should be associated with the citation or source detail. Lastly, there is an extract from a page added as the citation text.

Here is what part of that looks like:

Family Tree Maker 2009

0 @I00001@ INDI
	1 NAME Worth /Tucker/
	1 SEX U

Legacy 7

0 @I1@ INDI
	1 NAME Worth /Tucker/
		2 GIVN Worth
		2 SURN Tucker
	1 SEX U

RootsMagic 4

0 @I1@ INDI
	1 NAME Worth /Tucker/
		2 GIVN Worth
		2 SURN Tucker

These next lines start an individual (INDI) record (notice the 0 prefix). The ID between @ characters (ex: @I00001@) uniquely identifies this person record from any other person records in the file. The name of the person is indicated in one or two ways. The NAME tag requires the full name with the surname between “/” characters. The name can also be broken in given name and surname as represented by GIVN and SURN. Some formats set the gender or sex to U for unknown or don’t include the tag unless it is set.

Included in the individual record is the property ownership event:

Family Tree Maker 2009

1 EVEN 80 acres
	2 TYPE Property
	2 DATE 1908
	2 PLAC Elmo, Emery, Utah

Legacy 7

1 EVEN 80 acres
	2 TYPE Property
	2 DATE 1908
	2 PLAC Elmo, Emery, Utah

RootsMagic 4

1 PROP 80 acres
	2 DATE 1908
	2 PLAC Elmo, Emery, Utah

FTM and Legacy choose to use generic event tag with a corresponding qualifying type of “Property” whereas RM simplifies it by using the property tag. Both are equivalent. Each includes the property description of “80 acres” and the date and place of ownership.

GEDCOM supports the concept of a master source with its information and multiple citation details corresponding to an event. In part 1, this concept was shown in the application screens in two screens (FTM 2009 and Legacy 7) or a single screen with different colored top and bottom sections (RootsMagic 4). The way this is represented in the GEDCOM format is to have a separate source record with a unique id and to reference that source record for the specific event being cited. Additional citation details are then given. Notice that the below section starts at level 2 and appears in the file at the same level as PLAC and right below it. This signifies that the event being cited is the parent level 1 or the property ownership event.

Family Tree Maker 2009

2 SOUR @S00002@
	3 PAGE 179
	3 DATA
		4 TEXT In 1908 Eliza Oviatt filed on eighty acres and Worth Tucker purchased
			5 CONC eighty acres of an adjacent school section. These properties became
			5 CONC the Elmo townsite, platted into lots that were sold to prospective
			5 CONC residents for $10.

Legacy 7

2 SOUR @S4@
	3 PAGE 179.
	3 DATA
		4 TEXT In 1908 Eliza Oviatt filed on eighty acres and Worth Tucke
			5 CONC r purchased eighty acres of an adjacent school section. The
			5 CONC se properties became the Elmo townsite, platted into lots t
			5 CONC hat were sold to prospective residents for $10.
	3 OBJE
		4 FORM jpg
		4 FILE C:\Users\mtucker\Documents\RootsMagic downloads - test\historyofemerycounty-page179.jpg
		4 _SCBK Y
		4 _PRIM Y
		4 _TYPE PHOTO

RootsMagic 4

2 SOUR @S1@
	3 PAGE 179
	3 DATA
		4 TEXT In 1908 Eliza Oviatt filed on eighty acres and Worth Tucker purchased e
			5 CONC ighty acres of an adjacent school section. These properties became the E
			5 CONC lmo townsite, platted into lots that were sold to prospective residents f
			5 CONC or $10.
	3 OBJE
		4 FORM jpg
		4 FILE C:\Users\mtucker\Documents\RootsMagic downloads - test\historyofemerycounty-page179.jpg
		4 _SCBK Y
		4 _PRIM Y
		4 _TYPE PHOTO
	3 _TMPLT
		4 FIELD
			5 NAME Page
			5 VALUE 179

The source tag (SOUR) here references the source record with the unique id between the “@” characters. We will look at its details in the next section. The level 3s and higher are for the citation detail. This citation is from page 179 of the source and text from that page is represented by the TEXT tag under the DATA tag. The concatenation tag is used to break up lengthy text onto multiple lines. Both Legacy 7 and RootsMagic 4 export the directory path to the image file of page 179. This is done with the object (OBJE) tag which specifies that the format of the file is jpg. The next three tags begin with an underscore character “_” which means that they are custom extensions to GEDCOM made by applications and that other applications are not required to support them. It appears that both Legacy and RootsMagic have chosen to support these tags. The _SCBK tag indicates if this image should appear in the scrapbook or media viewer inside the application with a Y value signifying “yes”. The _PRIM tag indicates if this is the primary image which is shown in the application in situations where only one image about the citation is shown. The _TYPE tag identifies this as being a photograph as opposed to some other media type. RootsMagic 4 has done something above and beyond the other vendors. It supports a custom template tag (_TMPLT) which we will investigate in a minute.

Before we continue it might be helpful to review page 646 from Evidence Explained to see how the Book: Basic format is represented as a source list entry, full reference note, and short reference note:

Take just a minute to examine the fields for each type, formating, as well as the ordering and contents of the author field.

The final section of the GEDCOM file that we will examine is the actual source record:
Family Tree Maker 2009

0 @S00002@ SOUR
	1 TITL Geary, Edward A., A History of Emery County
	1 NOTE
		2 CONC Geary, Edward A..  A History of Emery County:  .  Salt Lake City:
		2 CONC Utah State Historical Society, 1996.

Legacy 7

0 @S4@ SOUR
	1 ABBR History of Emery County
	1 TITL A History of Emery County
	1 AUTH Edward A. Geary
	1 PUBL Salt Lake City: Utah State Historical Society, 1996.
	1 OBJE
		2 FORM jpg
		2 FILE C:\Users\mtucker\Documents\RootsMagic downloads - test\historyofemerycounty-frontcover.jpg
		2 _SCBK Y
		2 _PRIM Y
		2 _TYPE PHOTO
	1 OBJE
		2 FORM jpg
		2 FILE C:\Users\mtucker\Documents\RootsMagic downloads - test\historyofemerycounty-titlepage.jpg
		2 _SCBK Y
		2 _TYPE PHOTO
	1 OBJE
		2 FORM jpg
		2 FILE C:\Users\mtucker\Documents\RootsMagic downloads - test\historyofemerycounty-copyrightpage.jpg
		2 _SCBK Y
		2 _TYPE PHOTO

RootsMagic 4

0 @S1@ SOUR
	1 ABBR History of Emery County
	1 TITL Edward A. Geary, A History of Emery County (Salt Lake City: Utah S
		2 CONC tate Historical Society, 1996), [Page].
	1 _SUBQ Edward A. Geary, A History of Emery County, [Page].
	1 _BIBL Edward A. Geary. A History of Emery County. Salt Lake City: Utah S
		2 CONC tate Historical Society, 1996.
	1 OBJE
		2 FORM jpg
		2 FILE C:\Users\mtucker\Documents\RootsMagic downloads - test\historyofemerycounty-frontcover.jpg
		2 _SCBK Y
		2 _PRIM Y
		2 _TYPE PHOTO
	1 OBJE
		2 FORM jpg
		2 FILE C:\Users\mtucker\Documents\RootsMagic downloads - test\historyofemerycounty-titlepage.jpg
		2 _SCBK Y
		2 _PRIM N
		2 _TYPE PHOTO
	1 OBJE
		2 FORM jpg
		2 FILE C:\Users\mtucker\Documents\RootsMagic downloads - test\historyofemerycounty-copyrightpage.jpg
		2 _SCBK Y
		2 _PRIM N
		2 _TYPE PHOTO
	1 _TMPLT
		2 TID 372
		2 FIELD
			3 NAME Author
			3 VALUE Edward A. Geary
		2 FIELD
			3 NAME Title
			3 VALUE A History of Emery County
		2 FIELD
			3 NAME SubTitle
		2 FIELD
			3 NAME PubPlace
			3 VALUE Salt Lake City
		2 FIELD
			3 NAME Publisher
			3 VALUE Utah State Historical Society
		2 FIELD
			3 NAME PubDate
			3 VALUE 1996

FTM takes a shortcut and using the format specified for source list entry, puts that as the source note. The title (TITL) comes from how FTM automatically named the source when it was created. There are two things to note about the NOTE text: 1) it does not have a way to indicate that the book title should be italicized and 2) there appears to be extra periods and spaces in it.

Legacy 7 takes the approach of trying to stuff the EE-style citation into the fewer fields available for the “old-style” citation. There are 6 parts of a source list entry citation for a basic format book:

1. Author
2. Main Title
3. Sub Title
4. Place of publication
5. Publisher
6. Year

Legacy 7 implements the basic book template with the following fields (the last two pertaining to the citation detail):

1. Author Last Name
2. Author Given Name(s)
3. Author Suffix
4. Title
5. Short Title
6. Publisher City
7. Publisher State
8. Publisher
9. Publish Date
10. Volume Data
11. Page
12. Volume

The standard fields available in GEDCOM are:

1. Title
2. Author
3. Publication

So Legacy makes the following matches:

• GEDCOM Title = Title
• GEDCOM Author = Author Given Name(s) + Author Last Name + Author
• GEDCOM Publication = Publisher City + Publisher State + Publisher + Publish Date

Note that there is some formatting according to the source list entry and can be seen in the PUBL tag as it follows the order of fields and contains the colon and comma in the correct location. Also, the abbreviation (ABBR) tag is used to name the source in the master list after it is imported. The rest of the GEDCOM contents from the Legacy 7 file specify the 3 media files associated with the source. Nothing new there.

I have yet to do additional experiments to determine how the translation to “old style” citations works with more complicated citation formats.

Finally we look at RootsMagic 4. It also uses the abbreviation and object tags in the same way as Legacy 7. But some interesting things are happening in the rest of the file. Notice that the title tag follows the format for a full reference citation complete with parenthesis, commas, and colons. The title is between special formatting tags < i > and < /i > to indicate that it should be italicized. Where the page number should go is the textual placeholder “[Page]”. The custom subsequent tag (_SUBQ) contains the short note format although it should just contain the author’s last name. The custom bibliography tag (_BIBL) contains the source list entry format. It appears that a bug in the export is causing the bibliography entry to not show author with last name first. It is important to note that any application that imports a RM4-generated GEDCOM will get only the contents of the title tag and will have to manually edit it to remove the italicization indicators which they don’t support.

Now let’s get to the part where RootsMagic 4 has shown some innovation in their GEDCOM. Remember the custom template (_TMPLT) tag we saw for the citation:

3 _TMPLT
	4 FIELD
		5 NAME Page
		5 VALUE 179

There is also one in the source:

1 _TMPLT
	2 TID 372
	2 FIELD
		3 NAME Author
		3 VALUE Edward A. Geary
	2 FIELD
		3 NAME Title
		3 VALUE A History of Emery County
	2 FIELD
		3 NAME SubTitle
	2 FIELD
		3 NAME PubPlace
		3 VALUE Salt Lake City
	2 FIELD
		3 NAME Publisher
		3 VALUE Utah State Historical Society
	2 FIELD
		3 NAME PubDate
		3 VALUE 1996

Now compare that with the source entry screen:

Notice that in the yellow Master Source section, there are 6 entry fields: Author, Title, Sub-title, Publish Place, Publisher, and Publish Date. These correspond to the 6 template (_TMPLT) field name entries in the GEDCOM: Author, Title, SubTitle, PubPlace, Publisher, and PubDate. In the green Source Details section Page corresponds with the field name entry is the citation section: Page. The value tags contain the actual value. That way the details of knowing individual fields and values is not lost. Completing this is the template id or TID tag that is a unique number used internally by RootsMagic 4 to always refer to this template. That is why you can never edit existing templates in RootsMagic 4.

Here are the details of the template for id 372 as shown in the Source Templates screen:

It is interesting that each field is given a type to indicate if it is a Name, Place, Date, or Text. This could come in handy in future situations. Imagine searching all sources not just for the text “White” but for all sources that contain a name that contains “White.” Searches like that would return more appropriate results.

So what have we discovered is that the current three applications that support EE-style templates do so slightly differently on the input side (part 1) and vary greatly when it comes to GEDCOM output. As it stands today much is lost in the GEDCOM export rendering rich citations into blobs of text. RootsMagic 4 solves this problem in a proprietary way using its own template id and template fields names.  Currently no real interoperability exists between these applications when it comes to EE-style source citations.
This post is already long enough and I will likely expound on my ideas in a follow-up post. But imagine the RM4 implementation standardized and universally accepted. What a world of interoperability that would open up!

There is so much to think about. What do you think?

For those who want to know more, here are the details.

We will first start with the QuickCheck models found in Evidence Explained. These models can be used by software developers as a feature specification:

This example from page 646 specifies that a basic book citation consists of 7 parts:

1. Author
2. Main Title
3. Sub Title
4. Place of publication
5. Publisher
6. Year
7. Page

It also indicates the format of the citation specifying where to put commas, colons, periods, and parenthesis as well as when text is italicized.

These QuickCheck models as well as other citation formats specified in EE (Evidence Explained) have been coded into Legacy 7, Family Tree Maker 2009, and RootsMagic 4.

Now even though each of these 3 desktop genealogy applications used EE for their spec and received clarification from their outside business analyst, Elizabeth Shown Mills, there are slight variations in the implementation by each. To prove this, here are comparisons of each application and how it deals with a book citation:

RootsMagic 4

Legacy Family Tree 7

Family Tree Maker 2009

Here is a comparison of these three applications showing different interpretations of the EE citation model:

So currently we have 3 applications that support EE and each are slightly different. What is needed is a standard that each can measure up against so as more applications support these citation formats, tests can be created to verify compatibility. Some sort of consortium needs to be created to discuss the current differences and come to a consensus of opinion.

Let’s say that process has already occurred and agreement has been reached. Now a standardized file format can be designed to handle the additional level of detail required for the citation models. In some ways this file format would serve a similar purpose as GEDCOM, but would be updated with the capability of handling more-detailed source citation and referencing media such as images and files.

When visiting a website that supports this new file format, the researcher will encounter a download button or link that references that file:

In the above example, the link references a file called book.cite. The .cite extension represents a file of a specific content type (called a MIME type) identified as: application/cite+xml. The important thing to know about this is that a .cite file can now be uniquely identified from other content types.

We will not discuss the actual structure of the file at this point. Details will be provided in a later post.

When the researcher clicks the link, the web browser (in this case Firefox) prompts us if we would like to save or open the file:

As can be seen from the screen, the file is correctly identified as a CITE file and picks the default application used to open the file, ClickCite Launcher.
Part of the prototype code that I wrote was an application called ClickCite Launcher. Its purpose is to intercept CITE files and pass the file along to an importer application. The ClickCite Launcher application would need to be installed on the researcher’s computer and part of the installation would make a file association between .cite files and ClickCite Launcher. This is how Firefox knows which application to use.

The launcher application and the file association are for a computer running Windows. I am not familiar with Mac computers in this regard, but it would surprise me if a similar capability was not available.

This launcher application is aware of all installed applications that support importing of CITE files and presents the user with a list of desktop genealogy applications:

After an application is picked from the list and Import is clicked, the launcher application would start the importer application and pass it the book.cite file. The launcher application would likely be open source software whereas each desktop application that supported CITE files would provide their own importer. In the video demonstration, the importer for RootsMagic 4 presented no user interface and just updated the database file for the most-recently opened database, test.rmgc. The developers of the importer could choose to show a user interface that might include a list of all previously created databases and allow the user to choose which database would be the target of the import.

The process is depicted in the following diagram:

1. Genealogist clicks download link which causes the browser to download the file onto his/her computer.
2. When the file is manually opened or opened by the browser when the Open option is selected, Windows runs the application associated with the extension. In this case, .cite is associated to the launcher application. The launcher receives the location of the downloaded CITE file.
3. When Import is clicked on the launcher, the importer for the selected genealogy software is started and passed the location of the downloaded CITE file.
4. The importer loads the CITE file and adds information to the application’s database file.

I hope this addresses some of the technical questions that I have been receiving.

Thanks again to Twitter for being my news source.  Reunion, a popular genealogy application for the Mac has now been released for the iPhone/iPod Touch.  Reunion for the iPhone has many features including multiple family files, 6-generation overview, family card display, multimedia, and sources.  This is not just a family data viewer, you can edit data in Reunion for iPhone and then sync it with Reunion for Mac.

Here are a few screen shots:

This iPhone app and others that will likely appear this year addresses prediction #7 in the post, 9 Genealogy Predictions for 2009.

In March 2008, I presented at the BYU Family History Technology Workshop.  The topic was: 10 Things Genealogy Software Should Do.  The presentation slides are now also available on SlideShare.

In a recent post, Bruce Buzbee teased the genealogy software community by smudging the text of a mysterious 4th tab.  In a way I hate to be the spoiler and steal Bruce’s thunder, but I can’t help myself.  Actually it is Bruce’s fault.  He threw down the gauntlet and issued the challenge.  Maybe after Bruce is finished getting mad at me, he will give me a copy of RootsMagic 4 for being so clever.

As some of you may know, I am a Software Architect for my day job so writing code is one of the things that I do.  We’ll I was able to take the sample screen shots of RootsMagic 4 and plug them into an algorithm which in simple terms I will call a “desmudger.”  Since the smudging was done using the Gaussian Blur algorithm that is common in most graphics programs, it wasn’t hard to find a code sample of that algorithm and modify it to come up with the reverse.  The cool thing is that my algorithm requires at least three image samples in order to triangulate correctly and thanks to Bruce, I had three different screens shots.  What good fortune for me!  Here you can see that each of the smudge samples is just slightly different requiring at least three for the algorithm to work.

For all the work I put into the preparation, it was a shame that it was over so quickly.  Maybe I should have thought ahead and at least programmed a drum roll into the simple application.  Anyway, when I saw what the text on the 4th tab was, I immediately had that “oh that makes total sense” feeling.  It is one of the most requested wishes of all the beginner genealogists that I have talked to.  Below is a link to an image of the tab area with the mystery tab revealed.  Click on it only if you really want to know what the tab is.  There is no going back after you click it.  Please don’t hate me too much Bruce.

Spoiler link – Click only if you really want to know

In the last few posts at RootsMagic Blog, Bruce has leaked some new features of the upcoming RootsMagic 4. He says he will start small and progressively reveal more. The buzz is starting and you can sense the excitement as you read the comments. Some readers are trying to guess what the new features will be while others are hoping that their wishlist items will be included. Not all of the new features hinted at in the screen shots are explained by Bruce, so I thought that I would get in on the fun and use my keen eye to try and figure things out.

I honestly have no inside information myself, so don’t take my guesses as fact.  I simply opened my copy of RootsMagic 3 and compared it to the screen shots of RootsMagic 4.  It was almost like trying to solve one of those puzzles you would find in the Highlights magazine.

The first thing that I noticed was a tiny down arrow to the right of the Add Person toolbar item:

This probably drops down the same Individual, Spouse, Parents, Child menu as in RootsMagic 3. This is a simple change but a better user experience as it indicates that there are choices to be made.
 

The next thing I noticed was that the Notes icon on the toolbar has been flipped:

Not really a new feature, but did anybody else notice it too?  This makes me wonder if Bruce received a lot of e-mails from users who were bothered by what they thought was a “backwards” image. 
The next to last icon on the toolbar is a tree icon that looks very similar to the tree on the new FamilySearch website:

It is already known that RootsMagic 4 will support the new FamilySearch (NFS) once the APIs are ready. This is probably just what the icon will look like. It is my understanding that RootsMagic has been working a long time with The Church of Jesus Christ of Latter-day Saints on implementing the NFS API. RootsMagic might just be the first desktop genealogy application to support it.

I noticed that on the Pedigree, Family, and Descendant tabs on the far left side is a long thin bar with an arrow:

My guess is that this would open a sidebar that contains the list of people in the database and maybe even allow for searching.

The big mystery so far is what is the fourth tab? In the screen shots, the tab next to Descendants is blurred out.

I’ve spent a few minutes looking at the size of the smudge and comparing it to a smudge I made for the word “Family.” My guess is that the smudged word contains 5-7 letters. So that excludes “Map”, but the word could be “Source” or “Person” or “Synch.”

Go ahead and post your guesses. You might be right in which case you will have bragging rights. If not, you can join the rest of us interested guessers.

Here are links regarding the presentation:

• Paper - 18 pages
• Slides - 29 slides
• Video (recorded later) - 17:30

Would you like to see these 10 ideas implemented in the genealogy software that you use?  How can these ideas be improved upon?  I encourage you to respond.

The following are mentioned in the article:

People/Organizations

• Bill Buxton
• William Gibson
• Elizabeth Shown Mills
• Dick Eastman
• Elizabeth Kelley Kerstens
• DearMYRTLE
• Geoff Rasmussen
• Guy Kawasaki
• The Board for Certification of Genealogists

Software/Sites

• FamilyLink.com
• FamilyPursuit.com
• Geni.com
• MyHeritage.com
• WeRelate.org
• WorldVitalRecords.com
• Footnote.com
• Ancestry.com
• Clooz
• Legacy Family Tree
• Family Tree Maker

Topics

• Ideas
• Innovation
• Genealogy Software
• Design 
• Genealogical Proof Standard (GPS)
• Source Citation

Books

• Sketching User Experiences: Getting the Design Right and the Right Design
• Evidence! Citation and Analysis for the Family Historian
• Evidence Explained: Citing History Sources from Artifacts to Cyberspace

I encourage you to support this publication with your subscription.

Here is the abstract from my paper:

Innovation in genealogy software starts with ideas that lead to better design. This paper discusses 10 things that genealogy software should do but currently doesn’t. It is a starting point for discussion among those in the genealogy community: family historians, software developers, and designers. It is a springboard for additional design ideas.

With only 20 minutes, it will be both fast and fun.  If you will be attending either the workshop or the conference, it would be great to meet you.

Check out the schedule for other topics that will be discussed.

Are there features you would like added to the software that you use? What if a brand new genealogy application were designed from scratch, what would you want it to do? What is on your wish list for genealogy software?

Here is a chance to share your ideas.

By way of a challenge, I would like to see at least 50 responses. The more ideas, the better. So don’t be shy. Add your wish list item.

Design is represented by a funnel showing that more ideas exist at the beginning of the phase than at the end. Much of this information can be found in “Sketching User Experiences: getting the design right and the right design” by Bill Buxton.

Genealogists as well as designers and developers must work together as part of the design process. In the early part of this process, it is important to generate as many ideas as possible. No idea should be held back as it might be a stepping stone to a much better idea. Ideas tend to generate more ideas. As the two-time Nobel Prize winner, Linus Pauling, once said:

The best way to a good idea is to have lots of ideas.

The diagram indicates that no matter how many great ideas enter the funnel, there will be less at the end. Not all ideas survive.

On page 144 of “Sketching User Experiences”, a slightly different visualization by Paul Laseau is presented which shows two opposing funnels: one for idea generation and the other for idea reduction. My modification of the diagram is as follows:

The process begins with a single idea or a few ideas. This leads to more ideas. Ideas are explored quickly and cheaply and can be discarded just as fast – easy come, easy go.

At some point choices need to be made and ideas need to be refined. After all, idea generation cannot carry on indefinitely. We must create something to ship. If we do this correctly, we won’t just have something we will have the right thing. Ideas are refined roughly at first and then with more granularity. More ideas might still surface but they are more fine tuning of existing ideas than radical new ones. More choices are made as we approach the final design. If you think of these two funnels superimposed, it’s not too difficult to visualize the single Design funnel represented as Phase 0.

To make this point a second time, designers, developers, and users (genealogists in our case) are involved in this process. Everyone’s ideas are important. Designers share their ideas and also guide the others through the process.

The subtitle of “Sketching User Experiences” is “getting the design right and the right design.” This process of working together to generate ideas and refine them into a final design is part of getting the “right design.” When this process is not followed or those who use the software are not involved, a design will result but quite possibly not one that will provide an effective, usable, and enjoyable experience.

This blog is a place where we can have a conversation about design and go through the process together.  I want to listen to your ideas.  All ideas are welcome as we are at the starting point of design.  We can change the world of genealogy software.  Innovation can happen.

In a future post, we will explore the technique of sketching and how it can be used to quickly capture and share ideas.

The previous post ended by asking a few questions.  One of these was: Do we really need designers anyway?

Bill Buxton would answer that we need designers. Just because we are able to add up our grocery bill doesn’t make us all mathematicians. There are principles and practices of design that must be learned and honed. I know that Carnegie Mellon University has an acclaimed Human-Computer Interaction Institute whose mission is to create effective, usable, enjoyable experiences with technology. Are graduates in this area finding their way into genealogy software development?

Both authors (Bill Buxton and Alan Cooper introduced in the previous post) share a similar view about design’s place before development with users contributing significantly to the process. Typically the development process is Design, Engineering, and then Sales. Design is shaped like a funnel to indicate that the number of ideas or concepts at the beginning of the phase is greater than those at the end. The arrows indicate involvement from the other teams in the design process. Bill also accounts for the need sometimes to do engineering (or in the case of software: programming) before design as an input into the design process:

Let’s discuss the advanced Research and Development team for a minute.  What is it purpose?  Sometimes there are enough unknowns that you must do something first to determine what you do and don’t know.  The question might be if something is technically feasible. Or maybe we want to try out a concept some people who will actually use the product.  There are three main places that I have seen this:  Google Labs, Microsoft Research, and FamilySearch Labs. What is the purpose of all those free applications that Google develops?  One reason it to try out concepts and explore possibilities?  The same goes for Microsoft and I would propose it is the same for FamilySearch Labs.  Some projects that started in Phase -1 have later passed through the other phases to become a real product.

Take the recent example from FamilySearch Labs.  The Pedigree Viewer prototype has recently been incorporated into Genetree (which I wrote about in this post). This same viewer in combination with the Life Browser is now part of another FamilySearch Labs project, called Family Tree that can be used with the new FamilySearch.

It appears that some organizations in the genealogy software field understand the importance of design and are taking advantage of current principles and practices.  I hope many others will see it too.  It is my desire to encourage innovation in genealogy software.  We need better experiences with the software.  It needs to help us more.  These types of experiences must be designed.  Working together as genealogists and family historians, software developers, user experience designers, and management is the way to improvement.  We need to better connect as a community.  We must share ideas and knowledge.  We need to care.

Ideas. Design. Experience. Innovation.  Are these words also buzzing around in your head?

Instead of singing:

Que Sera, Sera,
Whatever will be, will be
The future’s not ours, to see

I sang:

Que Sera, Sera,
Whatever will be, will be
The future’s not hard to see

This personal joke has been used many times since then and has never failed to deliver a cheerful effect.

I think that the author and design leader, Bill Buxton, would agree that the “future’s not hard to see.” In his book, “Sketching User Experiences: getting the design right and the right design,” Bill stresses the importance of looking at least 5 years down the road when designing user experiences and adds:

Now most people say that you cannot predict the future, much less five years out. They use this as an excuse for not making the effort, or even contemplating it. I believe that this reflects a lack of training, technique, or responsibility on the part of design or management. (page 209)

He then goes on to quote William Gibson from an NPR interview on 30 November 1999:

… the future is already here. It’s just not very evenly distributed.

Buxton gives two examples. The first computer mouse was built in 1964 but didn’t reach widespread use until about 1995 – 30 years later. The idea for the CD came around 1965 but it wasn’t until 25 years later in 1990 that the industry reached $1 billion. From idea, to design, to prototype, to first production, to ubiquity takes time.

Bill then makes this statement that I would like us to consider:

If history is any indication, we should assume that any technology that is going to have a significant impact over the next 10 years is already 10 years old. (page 215)

 This made me wonder if this could be applied to genealogy software. What has happened over the last 10 years that could affect the design and innovation of genealogy software over the next 10 years?

At first nothing came to mind, but then I thought of Elizabeth Shown Mills’ book, “Evidence! Citation and Analysis for the Family Historian” which was published in 1997. In 2007, the much expanded “Evidence Explained: Citing History Sources from Artifacts to Cyberspace” was released. We are now beginning to see specific examples of Elizabeth’s work showing up in genealogy software. In a podcast interview by Dick Eastman, Elizabeth Kelley Kerstens indicated that the software Clooz version 2.x was influenced by handouts she received from Elizabeth Shown Mills. Another example is from a podcast by DearMYRTLE where she interviews Geoff Rasmussen about Legacy version 7. In this interview, Geoff gives a sneak peek of one of the major new features which is source citation following the standard set by Elizabeth Shown Mills. There is still much work that needs to be done in this area such as online databases providing better source citations. I talk about this in my previous post about Jumping Curves. So you see, the future is not so hard to see.

Another area that I want to point out is the Genealogical Proof Standard which was also created in 1997 by the Board for Certification of Genealogists. Other than its use in the certification process, you don’t hear much about it. But this standard is useful to researchers of all levels to help them get as close to the truth as possible. At some point genealogy software designers and developers will realize this and incorporate it into future genealogy software.

There are probably many more examples of ideas, technology, and methodology that exists today that will help us better see the future of genealogy software.

What things should be added to this list?

As a genealogy community, I hope we aren’t thinking:

Que Sera, Sera,
Whatever will be, will be
The future’s not ours, to see

Share your thoughts as the future is ours to see.  I hope that is not hard to see.

Compare the tree diagrams for yourself:

MacFamilyTree 5 beta

Microsoft’s Family.Show

The latest updates to genealogy software appear to have at least a graphic designer (if not a user experience designer) on the team.

Are there any genealogy Rich Internet Applications today? The answer is yes. At least a few. Currently, they all are created using Adobe Flash.

It might surprise you that FamilySearch Labs is on th leading edge of genealogy Rich Internet Applications with these prototype applications:

Pedigree Viewer
panning, zooming
direct-line highlighting
ancestor/descendant view
search
GEDCOM import

Life Browser
add artifact (photo, record)
edit details
associate artifact as evidence
image representing potential timeline feature

Other genealogy sites using RIAs include:

Geni
panning, zooming
enter details, support for single birth, marriage, death date
intuitive interface for adding parents, spouse, siblings, and children
appealing design

MyHeritage
2D/3D perspective, animation to help flow through tree
enter details, support for multiple facts (birth, marriage, death, many more)
intuitive interface for adding parents, spouse, siblings, and children
appealing design

Desktop or Web Applications

Through my years as a software developer and architect, I have developed both desktop and web applications. When I started my career more than 13 years ago, I had the choice of working on mainframe systems or on Windows desktops. I chose Windows. Many years later, web applications were all the rage. One reason for this was their ease of deployment. After a few years of doing web development, there was a backlash of sorts as the types of applications that my team was building were too complicated for the web. The users complained of slow performance and were unsatisfied with the user experience. So the pendulum swung back in favor of desktop applications that (this time) were connected to a server to get the data.

Other developers have continued to build for the web and have employed techniques such as Ajax to build more interactive web applications. Users have always had a choice of web browsers (Netscape, Safari, Firefox, Internet Explorer, Opera, etc.) which has caused developers extra work as the browsers don’t function the same. As web applications become more interactive, these differences in browsers continues to cause difficulties.

One advantage of desktop applications is that they can take advantage of the processing power of the computer that runs them. On the other hand, one issue you get with desktop applications is which operating system (or platform) to target: Windows, Mac, or Linux. Some developers have been seeking for the solution that would give them the “hat trick” of running on all three.

Rich Internet Applications

The development world is changing with the concept of the Rich Internet Application (RIA) which mixes the best of web applications and desktop software. You get the ease of deployment of the web with the better user experience and performance of the desktop. Although not a tenet of RIA-development itself, some of the technologies used to develop RIAs also solve the cross-platform problem. These technologies include:

• Adobe – Flash/Flex/AIR
• Microsoft – Silverlight
• Sun Microsystem – JavaFX

At this point Adobe is leading the pack due to the large number of computers that have the Flash player installed. In a weekly column in June 2007, Robert X. Cringely talks about RIA technologies and specifically why Flash comes out ahead of Java:

The folks at Macromedia (now Adobe) saw some amazing shortcomings in other web-based execution systems and simply did it right. Java applets were fantastic with major shortcomings (huge Java runtime, poor performance, clunky and ugly interface, etc). Flash fixed all or most of those. And Flash does cross-platform so much better than Java ever did.

RIAs built with Adobe tools can be run inside any browser that supports the Flash player or even outside the browser using the Adobe Integrated Runtime (AIR) to support cross-platform deployment. I am very excited about what is currently available from Adobe.

Microsoft’s Silverlight is in the early stages with version 1.0 that was released on September 4, 2007. I can see good things ahead for this technology as I have been using its sibling-technology Windows Presentation Foundation for almost a year now.

I have to admit that I haven’t looked closely at JavaFX. Any discussion of RIAs wouldn’t be complete if we didn’t at least acknowledge that Sun Microsystems sees this technology as important. I will be keeping an eye on this technology as well.

Here are links to sites that include sample Rich Internet Applications:

• Adobe Labs – AIR Showcase
• Adobe Flex Developer Center – Samples
• Microsoft Silverlight Gallery
• Tim Sneath blog – 50+ Silverlight-based applications

Conclusion

Over the next few years we will see less distinction between desktop applications and web applications and more focus on the applications themselves. More application developers will choose to create Rich Internet Applications and this will also apply to genealogy software. The technologies will become more mature and their capabilities will increase. When this happens there will be more applications available across operating systems. Imagine your favorite genealogy application available for Mac, Windows, and Linux. Smartphones, PDAs, and other devices will likely support RIAs furthering the reach of your applications. This trend is exciting to me both as a software developer who wants to create genealogy software as well as a family historian who uses it.

My idea is handwriting/paleography tutor software that will systematically teach users how to read early handwritten documents. 

Inspiration:

• “Reading Early American Handwriting” by Kip Sperry
• Typing Software

Typing software systematically teaches correct fingering and allows for repetitive practice.  It breaks these down into lessons and records both speed and accuracy. What if there were similar software that would help us read, transcribe, extract, or abstract handwritten documents?

The preface to Kip Sperry’s book states:

“One of the best ways to begin a study of paleography is to read facsimiles of documents and then transcribe them word for word, letter for letter.”

The software would include the following features:

• Instruction on how to transcribe a document
• Handwriting styles (secretary hand, Italic hand, etc.)
• Different modules for different languages and time periods (Nineteenth Century American, Eighteenth Century American, Eighteenth Century British, etc.)
• Start with transcribing single letters to get to know different alphabets and styles, then single words, then word phrases, then sentences/lines, and finally on to paragraphs.
• Expansion of abbreviations and contractions (Alex^r [Alexander], Cha^s[Charles])
• Arabic and roman numerals
• Dates (7ber [September])
• Obsolete letter forms (long s, thorn, etc.)

I imagine this software running on my desktop.  It will track my progress through the different lessons and show me my accuracy, problem areas, and maybe speed.  I can go back to previous lessons.  It would also be interesting if I could run it as a tool tray application where it would periodically, pop up a little window that asks me for a short translation.  That way I can practise a little each day.

One possible user interface idea comes from the UK National Archives site:

An expanded idea on this could be a website where people contribute image fragments (that are not under copyright) as well as the translations.  These could be categorized by language, handwriting style (if known), time period, difficulty level and length.  The Handwriting Tutor software could then download these “online modules” for an almost inexhaustible source of practice material.

 This idea is open to anyone who wants to work on it.  Let me know when you are finished or if anyone comes across software that does this.  Maybe the software can be sold as a companion to Kip Sperry’s book.

Useful links:

• National Archives - Palaeography
• About.com: Genealogy – Deciphering Old Handwriting
• Cyndi’s List - Handwriting and Script

In the sample data included for the Windsor family the number for each range was:

• 0-19: 3
• 20-39: 13
• 40-64: 17
• 65+: 16

The graph doesn’t include the additonal 32 individuals that didn’t have a birth and/or death date.  I am not certain why the ages were grouped as they were.

This graph also acts as a filter for the people list.  Clicking on each bar shows those individuals in that age range.

It is interesting how they chose to include both statistics and navigation in the same control.

A few weeks ago when I installed Family Tree Maker 2008 RC1, imagine my surprise when I discovered that it was built with the same frameworks that I use at work.  I can run the software and understand how it is composed at the same time.  I find myself thinking, this is the Shell, and these are Workspaces and the SmartParts. 

The following CAB terms are useful for this discussion:

• Shell – the main window of the application that contains menus, toolbars, and Workspaces
• Workspace – a user interface container that holds SmartParts
• SmartPart – a section of the user interface devoted to a specific task
• Module – a deployable unit of code that can contain SmartParts or business logic; the Shell loads one or more Modules

Some of the benefits of building an application with CAB/SCSF are:

• Separation of Concerns – one team can build the Shell and many teams or developers can simultaneously build the business logic and SmartParts and later put them together.  Also developers can focus on their strengths which results in more productive software development.
• Modularity – user interface components and business logic can be packaged is a deployable unit called a Module.  To extend the software, just add another Module.
• Extensibility – there are many extensibility points built into CAB/SCSF besides the Module which allow better evolution of software over time.

Because the Shell is loosely coupled (connected) to its contents (SmartParts) and SmartParts are loosely coupled to each other, then the software is built to evolve with less development cost and less chance of breaking existing code.

What this means for those who use Family Tree Maker is that the software has been refreshed and will continue its long life.  In fact, I am hopeful that the development team can deliver many enhancements in the future because of their decision to build on these frameworks provided by Microsoft.

I am also excited with the prospect of having genealogy software that I can customize.  Besides the extensibility features built into CAB\SCSF, Family Tree Maker also has a Plugins menu which indicates that they plan to offer extensibility through additional companies or maybe a plugin community.

In Family Tree Maker 2008 RC1, there are no Plugins listed. 

I am awaiting news from The Generations Network about guidelines for Plugin development.  I will share them as I am able.

Links for Developers and Architects:

• Overview of the Composite UI Application Block
• Architecting Composite Smart Clients Using CAB and SCSF
• CABPedia

When you click on a name in the surname cloud, then the person list is filtered to only show those that have that last name.

I have yet to see a surname cloud in any other genealogy software.

The user interface is clean and consistent.  A tutorial explains the application and there are help links available on every page.  I was interested in how the company would fulfill their promise to “enable genealogy enthusiasts to involve family members who have never engaged in family history work. ”  After spending a few hours using the beta, I think Family Pursuit, LC is on a path of innovation for genealogy software.  Those more experienced with family history now have tools available to mentor beginners in the research process and work collaboratively toward a common goal.

It is early in the beta process and not all features are available.  Overall, I like what I have seen so far and will share more details as the beta testing continues.