Explorations in Photogrammetry – Part 2

In part 2 of my series on photogrammetry, I will discuss the process of creating the three-dimensional model of the bowl from the National Museum (for more information on the process of taking the images of the bowl—which I mentioned in my last post—see part 1 of Explorations in Photogrammetry).  The process itself involves the use of two pieces of software:  Adobe Photoshop and Agisoft Photoscan Pro.

Photoshop to the Rescue

The first step was to unify the images. This process was done using Photoshop and a few of its “auto” features. I began by setting up a batch job (learn how to create batch jobs in Photoshop here). This job applied the following auto corrections in order:

  1. Auto contrast – to adjust the brightness settings of each photograph and create a uniform brightness/contrast
  2. Auto colour – to correct any oddities in colour
  3. Auto tone – to smooth out an residual white / black in the image and apply a universal look

Each file was then re-saved in a separate location. It’s best practice to always keep a copy of the original image, unedited.

Building the Model in Photoscan Pro

The next step was to import the photos into Photoscan Pro.  Photoscan Pro is software that allows a number of images to be “stitched” together using a point cloud.  This point cloud is then used to construct a 3D model of the object.

The first step involved assessing the quality of the images.  Photoscan Pro has a built-in quality assessment tool.  After running this tool against all the images imported into the programme, each photo is then given a score (from 0 to 1) that shows how high the quality of the image is in order to align it and produce a 3D model.  While learning how to use the software in class, we were taught a general rule of thumb is to only use images whose quality is .6 or higher.

After importing the images and assessing their quality, I received the following results:

Number of Images Quality % of Total
7 .40 – .49 5%
45 .50 – .59 29%
101 .60 and above 66%

I was a little dismayed that such a large number of my images (over 30%) were under the .60 threshold.  I decided to run two different models in order to compare the results.

I began by excluding all images under .60 quality. I applied a mask to each image so as to instruct Photoscan to ignore everything other than object (this involved basically “cropping” out the background and having Photoscan ignore anything but the bowl object—a very time intensive process but well worth the results).  I then used Photoscan to align the photos, build a point cloud, a dense point cloud, and then a mesh.  This created a somewhat reasonable 3D model of the bowl; however there were a number of errors in its rendering and the model itself looked incomplete.

In order to correct this issue, I attempted to rerun the model by including the 45 images that were marked with an image quality of .50 – .59.  When including these images, along with the original 101 images of .60 quality or better, I received a much stronger model. This model lacked the errors in its rendering that were present in the first model and looked more complete.  Upon close inspection, however, one can see the very bottom of the bowl looks as though it was “cut out”.  This is not a feature of the bowl itself, but rather a failure to capture images at a deep enough angle so as to capture the entire inside of the bowl.

As I began to closely inspect the second version of the bowl, I also noticed there was quite a bit of “noise” in the model (areas of the bowl that looked pixelated or out of place).  I attempted to reapply the masks on the objects by cropping the images further (thus excluding the very edges of the object in each photo).  However, this did not turn out well. My bowl ended up rather flat-looking, as though someone had collapsed the bowl upon itself.  I think took the original point cloud from the second model and began manually cleaning up the cloud by looking for areas of white (which were the background and thus not part of the bowl).  After manually removing these points, I rebuilt the mesh and texture and developed my final model.

Assessment

Overall, the bowl turned out better than I initially expected, especially given this was my first attempt at such an endeavour.  Given the opportunity to repeat the process under the same settings, I would make the following corrections:

  1. Adjust the aperture and ISO. For most of the images, I was using an F-stop of 8 and an ISO of 640.  I think I would adjust this to lower the ISO (thus reducing visual noise in the image) and raise the F-stop on the camera.  The exact values I would use are difficult to say, but I would most likely try to lower the ISO to somewhere between 100 and 200 and raise the F-stop up to at least 22.  This would give me a crisper image, especially in regards to depth of field, as well as the added benefit of removing a small amount of noise that exists in the image.
  2. Adjust the shutter speed.  I left the shutter speed on the automatic setting for this exercise, mainly because I didn’t think I would need to adjust it.  Given another opportunity, I might attempt to manually adjust this setting to see how it affects the quality of light in the image (many of images seemed a little dark, despite all of the artificial lighting present).
  3. Adjust the angle.  I would also take another round of photographs at a much steeper angle so as to completely capture the interior of the bowl.
  4. Take more time.  As I was the first student to begin taking pictures, I self-imposed pressure to complete my pictures quickly, so as to ensure my fellow classmates had ample time with the camera as well.  If I repeated this process again, I would be less conscious of time constraints and take more time in between photos to evaluate the quality of the image and adjust my settings as needed.

Final Models

In an effort to show off the difference between the two different models, I’ve included both below, as well as a version of the bowl that was sent to the 3D Printer (more on 3D printing in a future blog post). The first is the “bad” model that only included the images with a quality of .60 or higher. The second model includes images with a quality of .50 or higher. Also note the absence of part of the bottom of the bowl where I failed to take images at a deep enough angle to cover the entire bottom of the bowl.  The model used by the 3D printer does not have the hole in the bottom of the bowl as I used 3ds Max to fill in the mesh.

“Bad” Bowl
“Good” Bowl
“Printed” Bowl

Acknowledgements

Special thanks to the National Museum of Ireland and their Archaeology department for allowing us to photograph some of their objects. The pictures taken of the bowl featured in this post and my previous post were courtesy of National Museum of Ireland – Archaeology. All rights reserved.

Coming Up Next…

In part 3 of my series on photogrammetry, I will discuss my choice for the second part of my assignment. I will detail why I chose the particular object in question and what challenges were presented to me as part of this aspect of the assignment. I will also detail the steps I took to overcome those challenges.

Designing the Diary

We are now almost two months into the second half of Digital Scholarly Editing. The bulk of the work this semester is focused on the creation of a digital scholarly edition. We have chosen the war diary of Albert Woodman, a signaller with the Royal Engineers during the Great War. The diary itself is an interesting object; it spans two physical books and, unlike traditional diaries in which the author tends to only write a single entry per page, Mr. Woodman will often have multiple entries on a single page (presumably to conserve paper). Additionally, he often inserts newspaper clippings and other miscellany into the diary, which is not easy to represent digitally.

Representing Related Media

One of the biggest questions we had to address was, “How do we represent these miscellaneous objects digitally in a way that holds true to the spirit of their analogue representation?” Most of these objects are directly tied to a diary entry (often, Mr. Woodman makes mention of the object in question in his entry, or the object itself refers to a battle or news item he discussed in a particular entry). Showing them separately or as secondary entries in the diary breaks the metaphor of the diary itself. After all, you can’t really have two entries for the same day—that isn’t in keeping with the way a user envisions a diary to work.

Ultimately we decided to tie these items to an entry as “related media”. From an implementation standpoint, this is relatively simplistic. Within the TEI of the diary, we simply insert another <div> tag at the bottom of the <div> day which wraps that day’s entry. This <div> tag is then given a type attribute with a value of “insert”. When we run the TEI through an XSLT transformation mechanism, these “related media” divs are then extracted and added to the entry in a section on the page titled accordingly.

As to the represented model—the model that bridges the gap between the user’s mental, or expected model[1]—and the implementation, it was decided the best implementation would be to group these additional inserts (which may also include other interesting bits of media we find relevant to an entry) and provide a lightbox implementation in order to view them. The lightbox, a modal popup which presents an image in an overlay[2], has numerous advantages:

  1. It provides additional screen real estate. By loading larger versions of the image into an overlay, a thumbnail of the image can be displayed on the main screen, which has a much smaller visual footprint.
  2. It can provide increased performance. Many lightbox implementations utilise javascript and AJAX (Asyncronous Javascript And XML) to load images only when they are requested by the user. This means the image is not loaded into the DOM (Document Object Model) until the image is actually requested, thus cutting down on the amount of data that is transmitted to the user’s browser. The less data that is transmitted, the faster the page will load.
  3. It maintains the visual narrative. Every interface tells a story and the goal of every interaction should be to supplement that story. If a user clicks on an image and the page is then reloaded to display a larger view of the image on another page, the narrative is broken because the user is moved away from the page. In order to re-enter the narrative, the user must use the back button in the browser. Anything that breaks the visual narrative runs the risk of breaking the entire experience for the user and thereby decreasing the overall “stickiness” of the website.

When used properly, the lightbox can provide a strong user experience and present the designer with additional screen real estate that would otherwise be unavailable. By using a lightbox approach here, we have managed to solve the issue of the ephemeral material as well as the problems presented with its presentation.

Multiple Entries on a Single Page

The second major interaction issue we had to address was the appearance of multiple entries on a single page. In many digital scholarly editions, the transcription is presented with the original image side by side[3] [4] [5]. This allows for a direct comparison between the transcription and the original text. However, such an implementation in the Woodman Diary is confusing. Often times, an entry may begin in the middle of the page, but as we are trained to read from top to bottom, the user would immediately begin scanning the image from the top and may be confused as to why the transcription doesn’t match the image—not immediately realising the transcription begins with text half way down the “page” in the image. Multiple methods were considered for handling this unique situation:

  1. The image would not be viewable side by side with the transcription. The user would read the transcription and could click on a thumbnail of the image that would then display only the image with no transcription. This method was discarded due mainly to the expected user interaction of comparing transcription and image side by side.
  2. We would attempt to position the transcription text on the page to match its position in the image. This, however, would require quite a bit of extra encoding as we would need to encode locations of text at given pixel points within the original image. While a novel approach, it was ultimately decided this would require too much effort, given we are working with limited time and resources. Additionally, we felt it was a less aesthetically pleasing interaction.
  3. We would present multiple entries on a single day to match whatever was displayed in the related images. This idea was also discarded due to potential confusion by the user. Because the user has certain expectations in their mental model as a result of the diary metaphor, a user clicking on 25 January would expect to see one entry for 25 January. Under this model, however, they would instead receive an entry for 24 January and 25 January, which might be confusing. The break in the mental model is potentially jarring enough as to disrupt the visual narrative.

After considerable deliberation, it was decided to modify the third approach and create a hybrid. On the intial view of the diary entry, the user would see only the transcription of the entry he or she had selected. Thumbnails of the original diary page(s) would be presented, which could then be clicked and viewed in a lightbox. This lightbox would present a larger view of the image along with the transcription of that entire page. If text exists in the transcription that is not part of the entry being viewed, it would be rendered in a grey font indicating it is unrelated to the entry as a whole. As an example, if the user views an entry on 25 January, that entry begins in the middle of the page in the actual diary. When viewing that page in the lightbox, the latter half of the entry from 24 January is transcribed along with the entry for 25 January that appears on that image of the diary page. However the text for 24 January is rendered in a light grey so as to visually indicate to the user that it is unrelated to the entry being viewed, while still providing the user with the visual cue that the text he or she may be looking for in the image is not directly at the top of the image.

Conclusion

The Albert Woodman Diary has been an interesting project to handle. It has challenged the class as a whole to consider different ways of presenting an analogue object in a digital environment. By drawing on our own digital experiences as well as research conducted within the field of User Experience design, we have been able to overcome these challenges and present a true digital edition that adheres to the underlying metaphoric premise of the diary but without limiting the interactions by adhering to the metaphor too strictly.

References

[1] Cooper, Alan, Robert Reimann, and Dave Cronin. About Face 3: The Essentials of Interaction Design. Indianapolis: Wiley Publishing, 2007. Print.
[2] Adam. “Are Lightboxes Good Web Design?”. PugetWorks.com. 29 January 2011. Web. 20 March 2015.
[3]Sutherland, Kathryn., et al. Jane Austen’s Fiction Manuscripts. 2015. Web. 20 March 2015.
[4] Baillot, Anne.Letters and Texts: Intellectual Berlin around 1800. 2013. Web. 20 March 2015.
[5] Schreibman, Susan, et al. The Thomas MacGreevy Archive. 2001-2004. Web. 20 March 2015.

Explorations in Photogrammetry – Part 1

Our first assignment in AFF 621: Remaking the Physical involves photogrammetry.   Photogrammetry is a technique used to create three dimensional images of an object.  This is accomplished by taking a series of photographs of an object, where each photograph overlaps by at least 60%.  Special software is then used to create point clouds that “stitch” the pictures together into a three dimensional representation of the object.[1] 

Over a series of several blogs, I intend to document the process of photogrammetry used for my photogrammetry assignment.  The assignment itself calls for the creation of two different 3D models using photogrammetry. One of the models is to be a bowl housed at the National Museum of Ireland where the pictures can be taken in a controlled environment. The second model is to be one of my own choosing; however it must be taken in an outdoor setting where little environmental controls exist, thus giving us the opportunity to experiment with lighting, aperture, shutter speed, etc. The blogs will cover the following:

  • my trip to the National Museum of Ireland in Dublin
  • my choice of topic for the second model and the process involved in the actual photography
  • my experiences with the editing and creation of the actual models (2 separate posts)
  • an analysis of 3D printing and the service I would use to print my object

My Day at the Museum

For the first part of our assignment, we visited the National Museum of Ireland.  This was a fantastic opportunity to not only visit the museum, but to see some of the artefacts up close.  We spent the entire day in the museum photographing a number of bowls which dated from some time around 2,000 BCE. The Bronze Age era bowls themselves were largely used as drinking vessels and likely held some kind of beer.  Many of the bowls have a sun pattern on the bottom that some archaeologists speculate was tied to the worship of the sun by the indigenous people[2].

The process of photographing these objects was quite different than it was for photographing the second part of my assignment (the latter of which will be detailed in a future blog post).  As we were in a controlled environment, we were able to avail ourselves of a number of lighting techniques that wouldn’t be feasible outside such a setting.  One of these (which is also a personal favourite) was the lightbox.  lightbox The lightbox is white canvas-type cube that, as one of my classmates pointed out, looks much like a collapsible laundry basket. The object is then placed inside the lightbox and lights are placed around the outside.  The material of the lightbox acts as a sort of diffuser, softening the light and eliminating shadows. The image to the right shows the final setup of our lightbox.  The lights are positioned from all angles in order to provide the most light. This includes not only from the sides and front but also from above.  Once the object is placed on the turntable (which is positioned in the centre of the lightbox), the object can be slightly rotated in between each picture, allowing for the creation of the overlap between images necessary to produce an accurate model using photogrammetry.

Once everything is set up, the actual photography can begin.  As I mentioned above, the object is placed on the turntable within the lightbox, which allows the object to be easily rotated. Shane Using the EOS Utility (software provided by the Canon website, the manufacturer of the digital camera we used), we were able to preview each image on the laptop and adjust various settings as necessary.  Primarily I worked with the aperture (to define not only the light available but also clarity of the background and depth of field) as well as the ISO (which can allow for more brightness but must be used carefully so as not to introduce any visual “noise” into the image).  I decided to leave the shutter speed set to automatic, as adjusting this is more of an advanced technique and given the controlled environment, I didn’t feel it was necessary to adjust.  Once all of the adjustments were made, the process of photographing the images was quite simple.  Very few adjustments needed to be made in between images, and I was largely able to move from one image to the next with little intervention with the camera.

I decided to take images from four different angles, beginning at straight on, and then adjusting the angle of the camera up in order to capture further detail. The bowl was then flipped, and the process repeated in order to capture the bottom part of the bowl.  I was able to capture approximately 150 images in about 50 minutes (excluding the initial setup of the lightbox).

Acknowledgements

Special thanks to the National Museum of Ireland and their Archaeology department for allowing us to photograph some of their objects. The pictures taken of the bowl featured in this post and my previous post were courtesy of National Museum of Ireland – Archaeology. All rights reserved.

Coming Up Next…

In my next blog, I will discuss the outcome of these images, the steps taken to clean up the images for contrast, brightness, clarity, etc., and the process of utilising Photoscan Pro to create the 3D model.  I will also assess the quality of the images taken and what steps, if any, I would take in the future to ensure a higher quality of image as well as what lessons I can take away from this aspect of the assignment.

References

[1]US Bureau of Land management Publication: Tech Note 248 on Photogrammetry.

[2]Flanagan, Lauren. Ancient Ireland: Life Before the Celts. Gill & Macmillan, Ltd. 1998.