The Polymath Project began in February 2009 as an experiment to collectively work on an unsolved mathematics problem using a blog. Timothy Gowers and Michael Nelson started this project, and wrote this article in Nature, to test the success of an open-source platform to group problem solving. They wanted to prove the density Hales–Jewett theorem (DHJ), which while is known to be true still needs proofs.
The project was successful and in just over a month, the problem was solved. There were many contributors from all aspects of the mathematical community, such as an award winning researcher and a high school teacher. Gowers and Nielson liked the way their experiment differed from other team projects in the sciences, since in those teams "work is usually divided up in a static, hierarchical way. In the Polymath Project, everything was out in the open, so anybody could potentially contribute to any aspect."
Farsightedly, Nielson and Gowers realize this situation raises authorship and preservation issues. Who gets credit for solving the DHJ theorem? The authors of this article will provide a link to the blog as a list of collaborators, so that someone who didn't provide insightful comments won't get the same credit as another who made major advances. Of course, this brings preservation into it, since the blog might not always be a permanent link. Nielson an dGowers mention how the Library of Congress saves certain legal blogs and hope to emulate that somehow. I think the issue is more complex than that, and more investigation should be done on how other web sites are preserved.
The articles notes that "outside mathematics, open-source approaches have only slowly been adopted by scientists" but the authors give several examples for how other sciences could benefit from this kind of collaboration and online data sharing. Almost summing up what we've learned in class, Nielson and Gowers realize "the widespread adoption of such open-source techniques will require significant cultural changes in science." I'm not sure how a blog can be considered that revolutionary as 'open-source' , but opening up a process that is usually confined to classrooms and chalkboards is a step in the right direction. It's nice to see people within the community noting the problem and challenging their colleagues.
Wednesday, October 21, 2009
Furthering the Open Access Movement
In recognition of open access week, Jason Baird Jackson wrote a blog post titled Getting Yourself Out of the Business in Five Easy Steps. His five steps are as follows:
• Choose not to submit scholarly journal articles or other works to publications owned by for-profit firms.
• Say no, when asked to undertake peer-review work on a book or article manuscript that has been submitted for publication by a for-profit publisher or a journal under the control of a commercial publisher.
• Do not seek or accept the editorship of a journal owned or under the control of a commercial publisher.
• Do not take on the role of series editor for a book series being published by a for-profit publisher.
• Turn down invitations to join the editorial boards of commercially published journals or book series.
In effect, he is advocating a boycott against for-profit journals. In theory this sounds good, but I have trouble seeing it play out well in practice. AS we have discussed many times, many of the people writing articles are doing so in part to fulfill tenure requirements. Those requirements include publishing to the top journals in the field which are almost exclusively for profit. While the idea behind a boycott is noble, it seems hard to believe that researchers would be willing to give up publishing in the top journals.
Stevan Harnad wrote a blog post in response to Jackson saying just that. But more interestingly, he explained that just such a boycott was threatened in 2000 among biological researchers. As expected, the journals ignored the threat and the biologists never followed through with the boycott. That's when they decided to try something new. They launched PLoS, an alternative open-access journal for the field.
PLoS is now one of the great success stories of open access, becoming known for its quality and becoming a leading journal.
But may other open-access journals have not been so successful. Harnad suggests the solution to open-access lies in the institutional repository mandating of all published works of the researchers at an institution being placed into the repository.
Again, I have my doubts that this will work as institutional repositories do not have a good track record. Even when researchers are required to place articles in their repository, they don't always do it. And many for-profit journals will not allow a final version of the article to be placed in the repository, leaving the researcher forced to place preprints into the repository.
Clearly the system we have for journals is not working, but with few exceptions, institutional repositories are not the answer. Currently, it's pretty rare for an open-access journal to be considered as good as the top for-profit journals. We still haven't found an appropriate strategy to further the open access movement, only strategies that have had mixed results in the past. It seems that it is time for something new, but what form would it take?
• Choose not to submit scholarly journal articles or other works to publications owned by for-profit firms.
• Say no, when asked to undertake peer-review work on a book or article manuscript that has been submitted for publication by a for-profit publisher or a journal under the control of a commercial publisher.
• Do not seek or accept the editorship of a journal owned or under the control of a commercial publisher.
• Do not take on the role of series editor for a book series being published by a for-profit publisher.
• Turn down invitations to join the editorial boards of commercially published journals or book series.
In effect, he is advocating a boycott against for-profit journals. In theory this sounds good, but I have trouble seeing it play out well in practice. AS we have discussed many times, many of the people writing articles are doing so in part to fulfill tenure requirements. Those requirements include publishing to the top journals in the field which are almost exclusively for profit. While the idea behind a boycott is noble, it seems hard to believe that researchers would be willing to give up publishing in the top journals.
Stevan Harnad wrote a blog post in response to Jackson saying just that. But more interestingly, he explained that just such a boycott was threatened in 2000 among biological researchers. As expected, the journals ignored the threat and the biologists never followed through with the boycott. That's when they decided to try something new. They launched PLoS, an alternative open-access journal for the field.
PLoS is now one of the great success stories of open access, becoming known for its quality and becoming a leading journal.
But may other open-access journals have not been so successful. Harnad suggests the solution to open-access lies in the institutional repository mandating of all published works of the researchers at an institution being placed into the repository.
Again, I have my doubts that this will work as institutional repositories do not have a good track record. Even when researchers are required to place articles in their repository, they don't always do it. And many for-profit journals will not allow a final version of the article to be placed in the repository, leaving the researcher forced to place preprints into the repository.
Clearly the system we have for journals is not working, but with few exceptions, institutional repositories are not the answer. Currently, it's pretty rare for an open-access journal to be considered as good as the top for-profit journals. We still haven't found an appropriate strategy to further the open access movement, only strategies that have had mixed results in the past. It seems that it is time for something new, but what form would it take?
Tuesday, October 20, 2009
How do policies help?
In chapter two of the NSF publication we read this week, The Elements of the Digital Data Collections, the NSF says that policy needs to be developed in order to support the life of digital data. This chapter does not exactly explain why policies for data collections are needed or what exactly they hope to accomplish with policies. Within the document the NSF mentions that "a “one-size-fits-all” approach to policy development is inadequate". So why are they still asking for people to work on creating policies?
Also this week I read the short blog entry William Uricchio and the object/subject in participatory media, which is about a talk from William Uricchio last week. This a cursory look at participatory culture and how our culture has developed systems based on the participation of millions of people. Two of the examples Uricchio provides are SETI and iPhone apps. Systems of participation exist without models for how they should operate, instead these tools rely simply on what was possible when they were being created and based on what the needs of the users and developers were. This is very much the case for the cyberinfrastructure tools we've seen this semester.
One difference between the NSF document and the Uricchio talk is that the NSF is continually looking for a monolithic answer to cyberinfrastructure while Uricchio provides examples of working systems that could inspire the cyberinfrastructure. In particular, there's Photosynth, a tool from Microsoft that creates 3D "experiences" based on collective digital photography. It is definitely worth downloading Silverlight to see how the photos have been crowdsourced and curated in a meaninful way. About this example, the blog author says "Rather than building a space by building a model, the model emerges from the production of thousands of amateur photographers". This is certainly something I've been thinking about in terms of the cyberinfrastructure - before deciding what it will be and what it should be called, the cyberinfrastrcuture should emerge as a production of collaborative effort. After all, there is no working model for the cyberinfrastructure.
After all the discussion in class about building the cyberinfrastructure and reading about the alledged need for policies that will shape the cyberinfrastructure, I'm a bit skeptical that focusing on the end result of policy is helpful. Any work that's done on developing policy may be premature until there are more applicable cyberinfrastructure tools. Also, I'm afraid of what kinds of developments could potentially be stifled by inadequate policies. And again, there is no "one-size-fits-all" answer to all digital curation projects, these projects arise based on very specific needs of the people who use the data.
The one policy issue that I think will be the most difficult to tame long-term is copyright. I have been thinking lately that the best way to shape these policies is to simply participate in fair use activities. My hope is that if enough people are asserting their right to fair use, policies will be developed and strengthened that reflect those behaviors.
Ultimately, I wonder what policies will actually do for cyberinfrastructure. Why do the people involved want these policies? Do they simply desire a map to follow because that is the model they are familiar with?
Also this week I read the short blog entry William Uricchio and the object/subject in participatory media, which is about a talk from William Uricchio last week. This a cursory look at participatory culture and how our culture has developed systems based on the participation of millions of people. Two of the examples Uricchio provides are SETI and iPhone apps. Systems of participation exist without models for how they should operate, instead these tools rely simply on what was possible when they were being created and based on what the needs of the users and developers were. This is very much the case for the cyberinfrastructure tools we've seen this semester.
One difference between the NSF document and the Uricchio talk is that the NSF is continually looking for a monolithic answer to cyberinfrastructure while Uricchio provides examples of working systems that could inspire the cyberinfrastructure. In particular, there's Photosynth, a tool from Microsoft that creates 3D "experiences" based on collective digital photography. It is definitely worth downloading Silverlight to see how the photos have been crowdsourced and curated in a meaninful way. About this example, the blog author says "Rather than building a space by building a model, the model emerges from the production of thousands of amateur photographers". This is certainly something I've been thinking about in terms of the cyberinfrastructure - before deciding what it will be and what it should be called, the cyberinfrastrcuture should emerge as a production of collaborative effort. After all, there is no working model for the cyberinfrastructure.
After all the discussion in class about building the cyberinfrastructure and reading about the alledged need for policies that will shape the cyberinfrastructure, I'm a bit skeptical that focusing on the end result of policy is helpful. Any work that's done on developing policy may be premature until there are more applicable cyberinfrastructure tools. Also, I'm afraid of what kinds of developments could potentially be stifled by inadequate policies. And again, there is no "one-size-fits-all" answer to all digital curation projects, these projects arise based on very specific needs of the people who use the data.
The one policy issue that I think will be the most difficult to tame long-term is copyright. I have been thinking lately that the best way to shape these policies is to simply participate in fair use activities. My hope is that if enough people are asserting their right to fair use, policies will be developed and strengthened that reflect those behaviors.
Ultimately, I wonder what policies will actually do for cyberinfrastructure. Why do the people involved want these policies? Do they simply desire a map to follow because that is the model they are familiar with?
History of the Old and New
First thing I should say is happy Open Access week everyone. I am sure you have discovered in your perusals of the interwebs that this week is set aside for the celebration of free information for all. I recommend you take time to explain to your loved ones the value that open access has in their lives.
In the non-open access side someone posted to the insider about a historical document site called Footnote, I did not immediately investigate but I put it on the back burner to check out later. Then a few minuets later when I was reading a blog Footnote came up yet again. I felt that I must look into it.
Footnote is a semi-free, (read mostly pay) historical document image dataset. There are all sorts of interesting things to explore, News Papers, Texas Death records, Records from WWII, the founding of Liberia and more. Some 50 million images or so the site claims.
The viewer is really nice. The images are clear (better than Google books), the zoom is good and the interface is pretty easy. The ability to annotate images is very interesting but I am not sure how that works; if the annotations are just for you or for everyone. Users can create connections between documents and images that they upload. Did I mention that users can up load images? You can upload images and create pages about people or places or events. Interesting not all of these items are “historic” in the proper term. These are very odd. You can also print copies of the documents. Which I did… There is yet more functionality that I have not had the time to explore. This has the potential to be a very powerful cultural heritage tool.
Now here are my caveats. The free version is highly restrictive. It seems like ever document I tried to query on came up with a premium page notice. This is kind of unfortunate because I really wanted a chance to explore the scope of documentation. Many of the documents come from NARA or the LOC and I understand that what the site is selling is the “service” but still I would like the opportunity to play a bit more before I buy.
Another caveat is provenance. And I guess this is the age old question, how do you trust what you find on-line. I assume that there is some sort of community in place here that cleans the documents and ensures that the information is valid and accurate. Another question I have is how to ensure the contents move into the future.
This site is very interesting to me but I am not quite sure why. There is a mix here of the personal and the historical that you don’t usually see elsewhere. I really find it fascinating, it is an experiment with meaning and historical documentation that does not usually happen.
In the non-open access side someone posted to the insider about a historical document site called Footnote, I did not immediately investigate but I put it on the back burner to check out later. Then a few minuets later when I was reading a blog Footnote came up yet again. I felt that I must look into it.
Footnote is a semi-free, (read mostly pay) historical document image dataset. There are all sorts of interesting things to explore, News Papers, Texas Death records, Records from WWII, the founding of Liberia and more. Some 50 million images or so the site claims.
The viewer is really nice. The images are clear (better than Google books), the zoom is good and the interface is pretty easy. The ability to annotate images is very interesting but I am not sure how that works; if the annotations are just for you or for everyone. Users can create connections between documents and images that they upload. Did I mention that users can up load images? You can upload images and create pages about people or places or events. Interesting not all of these items are “historic” in the proper term. These are very odd. You can also print copies of the documents. Which I did… There is yet more functionality that I have not had the time to explore. This has the potential to be a very powerful cultural heritage tool.
Now here are my caveats. The free version is highly restrictive. It seems like ever document I tried to query on came up with a premium page notice. This is kind of unfortunate because I really wanted a chance to explore the scope of documentation. Many of the documents come from NARA or the LOC and I understand that what the site is selling is the “service” but still I would like the opportunity to play a bit more before I buy.
Another caveat is provenance. And I guess this is the age old question, how do you trust what you find on-line. I assume that there is some sort of community in place here that cleans the documents and ensures that the information is valid and accurate. Another question I have is how to ensure the contents move into the future.
This site is very interesting to me but I am not quite sure why. There is a mix here of the personal and the historical that you don’t usually see elsewhere. I really find it fascinating, it is an experiment with meaning and historical documentation that does not usually happen.
Research Remix
Open Access is one of the major issues that we keep revisiting in this class. In honor of International Open Access Week (which spans from October 19-23, and is an “international movement that pushes for broad and free access to research findings and publicly funded studies”), I’m blogging about the Issue Lab and its Research Remix, which I discovered at the Open Access Week website.
IssueLab is an open access archive and online publishing forum that gathers and shares nonprofit research about social issues. The mission of IssueLab is to “more effectively archive, distribute and promote the extensive and diverse body of work being produced by the third sector.” IssueLab aims to make it easier for nonprofits to manage and share their research with others.
On Monday, coinciding perfectly with the beginning of Open Access Week, the Research Remix was announced. This is a contest issued by IssueLab that calls for contestants to creatively reuse nonprofit research data to make videos that provide commentary on social issues. The goal of the IssueLab’s Research Remix is to get interested people to “apply their skills for social good, to base their social commentary in fact, and to learn about legally remixing and re-purposing existing content.”
At the IssueLab website there are over 300 (I believe the current count to be 335) Creative Commons openly licensed research reports archived that contestants can use when making their videos; these reports all have openly licensed video footage, images and music for video-makers to repurpose. This video contest “aims to engage working artists and digital media students” while at the same time “encouraging nonprofits to make their research more broadly available and usable through open licensing.”
The deadline for this competition is December 31, and one of the requirements is that the submitted contest videos must be openly licensed, which I think is a great requirement because fair is fair; after all, it wouldn't be right to use someone else's openly available materials, and then not let anyone else remix your own. All final videos will be made available to the organizations whose research was used in remixing.
I love the idea of the Research Remix and I think it will be something fun that inventive video-making types can really get involved with; as evidenced by the growing numbers of fan-made videos at websites like YouTube, people love to remix and repurpose things. By using openly licensed material, the Research Remix is a great way for people to make a statement about social issues while creatively expressing themselves legally. Legally is the key word in that sentence; much of the fan video remixing that is found online is illegal because users repurpose copyrighted material. I’m not sure how much this contest is really going to be encouraging for nonprofits to make their research more openly accessible, but I like that the thought is there and that an effort is being made.
That being said, I hope everyone is having a happy International Open Access Week.
IssueLab is an open access archive and online publishing forum that gathers and shares nonprofit research about social issues. The mission of IssueLab is to “more effectively archive, distribute and promote the extensive and diverse body of work being produced by the third sector.” IssueLab aims to make it easier for nonprofits to manage and share their research with others.
On Monday, coinciding perfectly with the beginning of Open Access Week, the Research Remix was announced. This is a contest issued by IssueLab that calls for contestants to creatively reuse nonprofit research data to make videos that provide commentary on social issues. The goal of the IssueLab’s Research Remix is to get interested people to “apply their skills for social good, to base their social commentary in fact, and to learn about legally remixing and re-purposing existing content.”
At the IssueLab website there are over 300 (I believe the current count to be 335) Creative Commons openly licensed research reports archived that contestants can use when making their videos; these reports all have openly licensed video footage, images and music for video-makers to repurpose. This video contest “aims to engage working artists and digital media students” while at the same time “encouraging nonprofits to make their research more broadly available and usable through open licensing.”
The deadline for this competition is December 31, and one of the requirements is that the submitted contest videos must be openly licensed, which I think is a great requirement because fair is fair; after all, it wouldn't be right to use someone else's openly available materials, and then not let anyone else remix your own. All final videos will be made available to the organizations whose research was used in remixing.
I love the idea of the Research Remix and I think it will be something fun that inventive video-making types can really get involved with; as evidenced by the growing numbers of fan-made videos at websites like YouTube, people love to remix and repurpose things. By using openly licensed material, the Research Remix is a great way for people to make a statement about social issues while creatively expressing themselves legally. Legally is the key word in that sentence; much of the fan video remixing that is found online is illegal because users repurpose copyrighted material. I’m not sure how much this contest is really going to be encouraging for nonprofits to make their research more openly accessible, but I like that the thought is there and that an effort is being made.
That being said, I hope everyone is having a happy International Open Access Week.
Digital Repositories and Sustainability
In two blog posts from earlier this year, Dorothea Salo mused on the issue of sustainability with regard to digital repositories.
In the first post, entitled (aptly) "Sustainability," Salo was inspired to think about the sustainability of digital repositories when she learned that arXiv is looking for different funding sources because Cornell is no longer willing to foot the bill. The question that this brings to Salo's mind has to do with role of libraries in the support of digital archives, particularly open access digital archives. She asks, "Are we [libraries] really memory organizations, or are we only memory organizations for print?" Salo seems to lean toward the belief that the library's role in the maintenance of digital archives should be larger. This sentiment reminded me of the article by Sayeed Choudhury and Timothy Stinson that I posted on a few weeks ago; they, too, envisioned the library as having a prominent role in the future of digital archives.
Salo also spends a bit of time in her post venting her feelings about irresponsible digital curation. She cites an instances in which searching for a long-term home for an item that originated outside of her institution she came across a repository that had been abandoned. This repository, Salo relates, seems to have been "hacked up" by two people in their spare time, and abandoned after the founders "got a couple of publications out of the attempt." Understandably, Salo finds this sort of behavior inexcusable, particularly because other won't attempt to establish repositories for this discipline while this one is still in existence. Such irresponsibility is, she says, a betrayal of trust.
The second post, called "Object lesson: when researchers run repositories," relates the story of the Mana'o anthropology repository. Dr. Alex Golub, founder of this repository, is no longer able to maintain it and is attempting to find an institution that will pick up it his stead. Although she considers the Mana'o repository a worthy project with the best of intentions (unlike the one cited in the previous post), Salo believes that any one-person effort of this type will eventually have to be rescued, and that it is irresponsible not to anticipate this from the inception. Salo again muses on the role of the library in rescuing such digital archives, naming librarianship's "continuing error" as the fact that "they have no infrastructure or plan in place for accomplishing these rescues."
Salo present us with two scenarios in which the sustainability of digital repositories is at issue. In the first case, the repositories failure was due to the negligence of its founders. In the second scenario, the well-intentioned creator of a repository came to a point at which he was no longer able to handle it, and had not planned for such an eventuality. The larger issue in both of these cases is who will pick up the slack. Salo, being an academic librarian, questions the library's role in the maintenance of digital archives and repositories. However, one could also put things in a larger context and ask what type of institution is best suited to be the custodian of the kinds of data repositories and archives that are emerging. Should the burden fall on libraries? Or are agencies like the Nation Science Foundation better suited to handle such large projects? Or should new entities be founded whose sole purpose is to create and curate digital repositories? Finding the best home for a particular digital curation project is one way to ensure its continued sustainability.
Computer program proves Shakespeare didn't work alone, researchers claim - Times Online
Computer program proves Shakespeare didn't work alone, researchers claim - Times Online
and
http://news.yahoo.com/s/time/20091020/us_time/08599193097100
Following a blurb on Yahoo! news, I found this neat article from Times Online, in conjunction with a Time.com article, discussing how a distinguished Shakespearean authority, Sir Brian Vickers, used plagiarism detection software to determine authorship of the history play Edward III.
Some background concerning this literary mystery: there's been a mystery for over 400 years as to whether Shakespeare co-authored Edward III with Thomas Kyd. The earliest copies of the play are unattributed and was published anonymously in 1596. The possibility of Edward III being penned, in part, by Shakespeare has been debated back and forth and was completely ignored by mainstream publication until 1997.
The plagiarism software, called Pl@giarism and developed at the University of Maastricht, was created to help educators catch cases of plagiarism. Pl@giarism operates by comparing all documents in a specified directory or by scanning the internet for files or works that match a phrase from the document being scrutinized.
Sir Brian Vickers, in his study of Shakespeare and other period works, came up with the concept of a "lingusitic fingerprint," in other words, playwrights often use the same patterns of speech. Vickers argues that, while it is established that a group or society can share common words three word phrases, it is pretty unlikely for a match of metaphors and unusual parts of speech to come up unless it was written by the same person due to the concept of the linguistic fingerprint.
Vickers came up with 200 matches when he tested Edward III with Shakespeare's work published before 1596. The average match rate is 20 matches. The 200 matches that Vickers came up with, in terms of relevancy, to use a database term, is a pretty strong indication that the provenance of four scenes in Edward III can be given to Shakespeare. Vickers tested a series of "recycled" phrases such as "come in person hither," or "thou art thyself" and whole phrases like "lilies that fester smell worse than weeds," and sure enough, there were some closely related matches in Edward III!
The article ends with Vickers stating his hope that further computer developments can be made in order to help prove provenance of major anonymous works and that the English literature field will come to utilize such handy materials in their study of text.
In the light of what we've been talking about in class and how this ties in with digital curation, I thought that these two articles and the project it discussed, proved to be a compelling example of how the cyberinfrastructure can transform and add a new dimension to research in the humanities and essentially put to rest some stubborn mysteries that have plagued the English academia.
and
http://news.yahoo.com/s/time/20091020/us_time/08599193097100
Following a blurb on Yahoo! news, I found this neat article from Times Online, in conjunction with a Time.com article, discussing how a distinguished Shakespearean authority, Sir Brian Vickers, used plagiarism detection software to determine authorship of the history play Edward III.
Some background concerning this literary mystery: there's been a mystery for over 400 years as to whether Shakespeare co-authored Edward III with Thomas Kyd. The earliest copies of the play are unattributed and was published anonymously in 1596. The possibility of Edward III being penned, in part, by Shakespeare has been debated back and forth and was completely ignored by mainstream publication until 1997.
The plagiarism software, called Pl@giarism and developed at the University of Maastricht, was created to help educators catch cases of plagiarism. Pl@giarism operates by comparing all documents in a specified directory or by scanning the internet for files or works that match a phrase from the document being scrutinized.
Sir Brian Vickers, in his study of Shakespeare and other period works, came up with the concept of a "lingusitic fingerprint," in other words, playwrights often use the same patterns of speech. Vickers argues that, while it is established that a group or society can share common words three word phrases, it is pretty unlikely for a match of metaphors and unusual parts of speech to come up unless it was written by the same person due to the concept of the linguistic fingerprint.
Vickers came up with 200 matches when he tested Edward III with Shakespeare's work published before 1596. The average match rate is 20 matches. The 200 matches that Vickers came up with, in terms of relevancy, to use a database term, is a pretty strong indication that the provenance of four scenes in Edward III can be given to Shakespeare. Vickers tested a series of "recycled" phrases such as "come in person hither," or "thou art thyself" and whole phrases like "lilies that fester smell worse than weeds," and sure enough, there were some closely related matches in Edward III!
The article ends with Vickers stating his hope that further computer developments can be made in order to help prove provenance of major anonymous works and that the English literature field will come to utilize such handy materials in their study of text.
In the light of what we've been talking about in class and how this ties in with digital curation, I thought that these two articles and the project it discussed, proved to be a compelling example of how the cyberinfrastructure can transform and add a new dimension to research in the humanities and essentially put to rest some stubborn mysteries that have plagued the English academia.
visualizing data on demand!
The National Geophysical Data Center makes available large amounts of data, divided into sub-sections based on terrain and source: land, marine, satellite, snow & ice, and solar-terrestial. The NGDC is part of the US Department of Commerce, the National Oceanic & Atmospheric Administration, the National Environmental Satellite, and the Data and Information Service. The goal of NGDC is “
can produce custom images, on request, for many of our databases.” This really amazes me! That the NGDC is not only providing datasets (for free!) but also offering to work with users of the sets to create useful images derived from these sets makes it an excellent example of providing data, and the tools to visualize that data.
Specifics in digital collection policies
Last year, in this post on the DCC blog, Chris Rusbridge wrote about an Australian national code that all universities signed regarding research data and primary sources. The code is quite specific, I was surprised that the term recommended for keeping data is only 5 years, however, as Rusbridge writes, since it is specific, it is stronger than no policy at all.
The policy also states where and how to keep the data (not just in the researcher's own computer) and outlines researcher responsibilities as well as repository requirements. We've talked about the importance of being specific both to the field that a digital curation project is targeted toward, and in policy language for data sharing. This strikes me as an important step in Australia to mandate data preservation, and to some extent sharing.
Rusbridge writes that he does not know of any other policies exactly like the Australian one, which mandates that, "Each institution must have a policy on the retention of materials and research data." Although it is commonly known that collection policies are important, but at the time of this post the author mentions only one other similar policy, from the UK, the UK Research Integrity Office's Code of Conduct and Policy.
The UK code is certainly a good idea, but in my view it loses ground to the Australian code on specifics. For example, while the Australian code specifies that institutions must hold the data as well as the researcher, the UK code vaguely writes, "This is a shared responsibility between researcher and the research organisation, but individual researchers should always ensure that primary material is available to be checked..." Of course they *should but sometimes they don't, as we have discussed.
Another example of the language waffling in the UK document, "In addition, it is always a good idea, even if it is not required, to seek advice from an institutional official." (if unsure about revealing a conflict of interest.) Compare this to the Australian code, "Researchers must foster and maintain a research environment of intellectual honesty and integrity, and scholarly and scientific rigour." followed by 7 bullet points including, "...cite awards, degrees conferred and research publications accurately, including the status of any publication, such as under review or in press."
Since the Rusbridge post is a year old, I retrieved some recent collection policies from institutions to look at. Successful projects (meaning projects with active links and not empty) often have their collection policies available, and the policies are specific but still flexible.
For example, the Internet Children's Digital Library Collection Policy (not research data but still a curation project I believe, and given as an example on NSF's website) states as a goal, "to create a collection of more than 10,000 books in at least 100 languages that is freely available to children, teachers, librarians, parents, and scholars throughout the world via the Internet;" This specifies how many and what kind of materials without limiting the project.
The UK Data Archive Collections policy provides criteria for selecting and acquiring collection sets as well, the summary of which is, "The UKDA acquires data for
four central purposes:
• archival preservation;
• secondary use and analysis for research;
• teaching and learning use;
• replication and validation of research."
I then wanted to look a little closer to home so I tried the DSpace instance at UT libraries. The introduction states only, "The Repository's purpose is to collect, record, provide access to, and archive the scholarly and research works of the University of Texas at Austin, as well as works that reflect the intellectual and service environment of the campus." The collection policy provides several guidelines on the structure and metadata, but each collection must write their own description of what materials should be included.
Here is the description from the UT Faculty/Researcher Works, "Peer-reviewed pre-print articles, published articles, technical reports, white papers, presentations, collections of digitized data, field notes, etc…"
Compare this to the California Digital Library collection development framework. The CDL is another federation of several institutions and collections, much as the UT library attempts to network many diverse collections. In their overall collection development document, they have dictated a balance between disciplines, and the policy also states that,
"Conventional collection development criteria are paramount and will be applied consistently across formats, including digital resources.
Any policy is better than none, but one that states specific goals without limiting future extensibility and, when necessary, mandates further collection policies from participating institutions, will probably mean a more stable project infrastructure.
The policy also states where and how to keep the data (not just in the researcher's own computer) and outlines researcher responsibilities as well as repository requirements. We've talked about the importance of being specific both to the field that a digital curation project is targeted toward, and in policy language for data sharing. This strikes me as an important step in Australia to mandate data preservation, and to some extent sharing.
Rusbridge writes that he does not know of any other policies exactly like the Australian one, which mandates that, "Each institution must have a policy on the retention of materials and research data." Although it is commonly known that collection policies are important, but at the time of this post the author mentions only one other similar policy, from the UK, the UK Research Integrity Office's Code of Conduct and Policy.
The UK code is certainly a good idea, but in my view it loses ground to the Australian code on specifics. For example, while the Australian code specifies that institutions must hold the data as well as the researcher, the UK code vaguely writes, "This is a shared responsibility between researcher and the research organisation, but individual researchers should always ensure that primary material is available to be checked..." Of course they *should but sometimes they don't, as we have discussed.
Another example of the language waffling in the UK document, "In addition, it is always a good idea, even if it is not required, to seek advice from an institutional official." (if unsure about revealing a conflict of interest.) Compare this to the Australian code, "Researchers must foster and maintain a research environment of intellectual honesty and integrity, and scholarly and scientific rigour." followed by 7 bullet points including, "...cite awards, degrees conferred and research publications accurately, including the status of any publication, such as under review or in press."
Since the Rusbridge post is a year old, I retrieved some recent collection policies from institutions to look at. Successful projects (meaning projects with active links and not empty) often have their collection policies available, and the policies are specific but still flexible.
For example, the Internet Children's Digital Library Collection Policy (not research data but still a curation project I believe, and given as an example on NSF's website) states as a goal, "to create a collection of more than 10,000 books in at least 100 languages that is freely available to children, teachers, librarians, parents, and scholars throughout the world via the Internet;" This specifies how many and what kind of materials without limiting the project.
The UK Data Archive Collections policy provides criteria for selecting and acquiring collection sets as well, the summary of which is, "The UKDA acquires data for
four central purposes:
• archival preservation;
• secondary use and analysis for research;
• teaching and learning use;
• replication and validation of research."
I then wanted to look a little closer to home so I tried the DSpace instance at UT libraries. The introduction states only, "The Repository's purpose is to collect, record, provide access to, and archive the scholarly and research works of the University of Texas at Austin, as well as works that reflect the intellectual and service environment of the campus." The collection policy provides several guidelines on the structure and metadata, but each collection must write their own description of what materials should be included.
Here is the description from the UT Faculty/Researcher Works, "Peer-reviewed pre-print articles, published articles, technical reports, white papers, presentations, collections of digitized data, field notes, etc…"
Compare this to the California Digital Library collection development framework. The CDL is another federation of several institutions and collections, much as the UT library attempts to network many diverse collections. In their overall collection development document, they have dictated a balance between disciplines, and the policy also states that,
"Conventional collection development criteria are paramount and will be applied consistently across formats, including digital resources.
- Establishing a coherent rationale for the acquisition of each resource, meeting faculty and student information needs
- Providing orderly access and guidance to the digital resources, and integrating them into library service programs
- Ensuring that the advantages of the digital resource are significant enough to justify its selection in digital format."
Any policy is better than none, but one that states specific goals without limiting future extensibility and, when necessary, mandates further collection policies from participating institutions, will probably mean a more stable project infrastructure.
Remediation & Early English Books Online
Kichuk, Diana. "Metamorphosis: Remediation in Early English Books Online (EEBO)." Literary and Linguistic Computing. (published June 18, 2007). Available at http://llc.oxfordjournals.org.ezproxy.lib.utexas.edu/cgi/content/full/fqm018v1 (Accessed October 13, 2009).
Out of morbid curiosity I had wanted to read an article written about a project that I had worked on, Early English Books Online (EEBO), and this article by Diana Kichuk explores the layers of remediation in the project's production of its digital archives of Early English Books. As Kichuk defines it, remediation is the "re-presentation of one medium in another" (2) as well as the "appropriation or re-purposing of old media in new media" (a definition she attributes to R. Grusin and J.D. Bolter's Remediation: Understanding New Media [2000]). EEBO provides an interesting case study because its associated projects involve several levels of remediation.
EEBO consists of an archive of images of over 100, 000 books printed in English between 1473-1700, and EEBO-Text Creation Partnership (EEBO-TCP)--the part of the project that I worked for--now offers hand-keyed searchable, reading texts for 25,000 of these titles. Access in both cases is through individual or institutional subscription. The project entails several levels of remediation. The majority of the images of the Early English books come from the Early English Books microfilm project, which was began in the mid-1930s to preserve copies of Britain's important early books, and the project continued in the years following the Second World War. Beginning in 1998, these microfilms were then digitized to produce EEBO's digital archive of images. The clear reading texts produced by EEBO-TCP are drawn from human transcriptions the digitized microfilm images. Accordingly, Kichuk asserts that EEBO is a surrogate of a surrogate, with remediation acting like a "distorting lens or opaque veil through which the scholar 'sees' the mediated Early English book" (6). Though Kichuk states that the collection is of "formidable scholarly valuable," her article emphasizes the need to recognize that that EEBO does not offer an exact copy of the original and that scholars must be better aware of the limitations entailed in remediation.
Kichuk sees ProQuest's decision to digitize the microfilm facsimiles rather than print copies as ultimately wise: digitizing print copies that it did not own would have been prohibitively expensive and slow. Moreover, the technological limitations present at the time the project was initiated would likely have meant that the original books would have to be dis-bound to be scanned, thus endangering the original artifacts.
There are, however, important sacrifices associated with this decision, resulting in content amputation and page distortion, among other things. The EEB microfilms generally do not include the endpapers or bindings of the books, and the pages were often cropped, resulting both in the loss of some marginalia and the misrepresentation of the physicality of the original book. The page curvature resulting from open book photography generates distortions in font appearance and darkens the gutters, affecting both legibility and accuracy of representation. Also, the resolution used by the EEB microfilms was too low for grayscale capture, so the images are bi-tonal black and white, which makes capturing any traces of color in the original works near impossible. Also, in the transition from microfilm to digital image, EEBO again sacrificed detail by further lowering its image resolution in order to ensure acceptable download transmission rates (the images are 440 PPI [pixels per inch], too low for a preservation quality facsimile).
(This figure, taken from the EEBO homepage (http://eebo.chadwyck.com/marketing/eebo_demo11_tcp.htm) offers a sample of what the EEBO interface and text images look like.)
As a digital codex, Kichuk notes that the project does not accurately reflect both the text and the physicality of the original--to achieve this would require applying the latest imaging technologies to the original books. It would be a different project carried out at a different time. In terms of improvements though, Kichuk would like EEBO (and the scholars who use it) to be more conscious of and clear about the loss remediation entails. She argues for the vendor guard to against claims of authenticity and identical-ness. She would also like to see EEBO include duplicate copies, which would be of use to bibliographers (this seems to me a slightly odd request given that EEBO fulfills so little of a bibliographer's needs).
Though I think the issues of remediation are fascinating, the questions that EEBO and Kichuk's article leave me with are ones about what it is we want from our digital surrogates. EEBO is a hugely important source of access to the content of rare early printed books that most users would never have the opportunity to access otherwise. EEBO-TCP adds further value by allowing readers to engage in full-text searches across over 25,000 titles. This said, it is not as useful a resource if you are interested in thinking about the book as object. Its images are not high-quality representations and it has only limited functionality to support this type of investigation. (For example, I had a friend who was interested in using EEBO to study investigate the significance of when blackletter was used as opposed to roman or italic fonts in early English books. Since EEBO-TCP doesn't specify font-type in its encoding and only records emphasis by noting the distinction between the normative font for the book and any "highlighted" font by using
Out of morbid curiosity I had wanted to read an article written about a project that I had worked on, Early English Books Online (EEBO), and this article by Diana Kichuk explores the layers of remediation in the project's production of its digital archives of Early English Books. As Kichuk defines it, remediation is the "re-presentation of one medium in another" (2) as well as the "appropriation or re-purposing of old media in new media" (a definition she attributes to R. Grusin and J.D. Bolter's Remediation: Understanding New Media [2000]). EEBO provides an interesting case study because its associated projects involve several levels of remediation.
EEBO consists of an archive of images of over 100, 000 books printed in English between 1473-1700, and EEBO-Text Creation Partnership (EEBO-TCP)--the part of the project that I worked for--now offers hand-keyed searchable, reading texts for 25,000 of these titles. Access in both cases is through individual or institutional subscription. The project entails several levels of remediation. The majority of the images of the Early English books come from the Early English Books microfilm project, which was began in the mid-1930s to preserve copies of Britain's important early books, and the project continued in the years following the Second World War. Beginning in 1998, these microfilms were then digitized to produce EEBO's digital archive of images. The clear reading texts produced by EEBO-TCP are drawn from human transcriptions the digitized microfilm images. Accordingly, Kichuk asserts that EEBO is a surrogate of a surrogate, with remediation acting like a "distorting lens or opaque veil through which the scholar 'sees' the mediated Early English book" (6). Though Kichuk states that the collection is of "formidable scholarly valuable," her article emphasizes the need to recognize that that EEBO does not offer an exact copy of the original and that scholars must be better aware of the limitations entailed in remediation.
Kichuk sees ProQuest's decision to digitize the microfilm facsimiles rather than print copies as ultimately wise: digitizing print copies that it did not own would have been prohibitively expensive and slow. Moreover, the technological limitations present at the time the project was initiated would likely have meant that the original books would have to be dis-bound to be scanned, thus endangering the original artifacts.
There are, however, important sacrifices associated with this decision, resulting in content amputation and page distortion, among other things. The EEB microfilms generally do not include the endpapers or bindings of the books, and the pages were often cropped, resulting both in the loss of some marginalia and the misrepresentation of the physicality of the original book. The page curvature resulting from open book photography generates distortions in font appearance and darkens the gutters, affecting both legibility and accuracy of representation. Also, the resolution used by the EEB microfilms was too low for grayscale capture, so the images are bi-tonal black and white, which makes capturing any traces of color in the original works near impossible. Also, in the transition from microfilm to digital image, EEBO again sacrificed detail by further lowering its image resolution in order to ensure acceptable download transmission rates (the images are 440 PPI [pixels per inch], too low for a preservation quality facsimile).
(This figure, taken from the EEBO homepage (http://eebo.chadwyck.com/marketing/eebo_demo11_tcp.htm) offers a sample of what the EEBO interface and text images look like.)As a digital codex, Kichuk notes that the project does not accurately reflect both the text and the physicality of the original--to achieve this would require applying the latest imaging technologies to the original books. It would be a different project carried out at a different time. In terms of improvements though, Kichuk would like EEBO (and the scholars who use it) to be more conscious of and clear about the loss remediation entails. She argues for the vendor guard to against claims of authenticity and identical-ness. She would also like to see EEBO include duplicate copies, which would be of use to bibliographers (this seems to me a slightly odd request given that EEBO fulfills so little of a bibliographer's needs).
Though I think the issues of remediation are fascinating, the questions that EEBO and Kichuk's article leave me with are ones about what it is we want from our digital surrogates. EEBO is a hugely important source of access to the content of rare early printed books that most users would never have the opportunity to access otherwise. EEBO-TCP adds further value by allowing readers to engage in full-text searches across over 25,000 titles. This said, it is not as useful a resource if you are interested in thinking about the book as object. Its images are not high-quality representations and it has only limited functionality to support this type of investigation. (For example, I had a friend who was interested in using EEBO to study investigate the significance of when blackletter was used as opposed to roman or italic fonts in early English books. Since EEBO-TCP doesn't specify font-type in its encoding and only records emphasis by noting the distinction between the normative font for the book and any "highlighted" font by using
tags, there's simply no way of searching for this short of examining each text.)I would argue, however, that in many cases consulting EEBO could help someone in determining which books to examine in person.
Ultimately, reading about EEBO foregrounds important questions about how we should prioritize financial feasibility, technical limitations, functionality, and speed of collection development in generating digital libraries and large curated collections. Though EEBO is more about access than preservation, the issues it raises overlap at least partially with the discussion of essential elements in this week's reading (Harvey 16). Though Ross Harvey's context is somewhat different, his question: "Is the value tied to the way the material looks? (Would it be lost or significantly degraded if the material looked different?)" remains instructive in thinking about EEBO. In developing such projects, we must decide which features of the original are most important to capture and at what cost, while keeping in mind that remediation entails both loss and gain.
Ultimately, reading about EEBO foregrounds important questions about how we should prioritize financial feasibility, technical limitations, functionality, and speed of collection development in generating digital libraries and large curated collections. Though EEBO is more about access than preservation, the issues it raises overlap at least partially with the discussion of essential elements in this week's reading (Harvey 16). Though Ross Harvey's context is somewhat different, his question: "Is the value tied to the way the material looks? (Would it be lost or significantly degraded if the material looked different?)" remains instructive in thinking about EEBO. In developing such projects, we must decide which features of the original are most important to capture and at what cost, while keeping in mind that remediation entails both loss and gain.
Monday, October 19, 2009
A Tale of Compromise
Jessica Branco Colati, Robin Dean, and Keith Maull, “Describing Digital Objects: A Tale of Compromise,” Cataloging & Classification Quarterly 47, no. 3 (4, 2009): 326-369.
This article reports the efforts Alliance Digital Repository (ADR), a consortial digital repository service for the Colorado Alliance of Research Libraries (Alliance), to create a standard descriptive metadata policy for their repository records. Members in the Alliance included twelve separate libraries at nine different academies and institutions, public and private. The goal was to support each library's community standards while also ensuring interoperability. This would facilitate a central repository.
Needless to say the scope for a project like this, which is charged with accommodating a dozen libraries each with their own practices and metadata (not to mention unique data both from within and without the library), expands at a rate equal to thought itself.
The ADR eventually pared down the libraries' metadata they had to consider to:
As they note Fedora does not have a user-friendly interface for submission and searching built-in (like DSpace) so they chose Fez as a configurable front end, adding that other interfaces could be attached to Fedora as needed.
MODS was chosen as the "normalizing schema," the one all records would be converted to at a minimum no matter what metadata scheme they had upon ingestion. The report gives a really helpful list of reasons why, and I think it's interesting that #1 is the simple fact that Fez and MODS "are proven partners: The Fez system was already using MODS widely throughout the system as the primary means of describing objects." They give other factors: the right granularity, crosswalks to the popular data-sharing DC, and plain familiarity, but it's notable that the tool in this case (Fez) partly determined yet another tool.
ADR determined minimal metadata field requirements with DLF Aquifier, a Digital Library Federation initiative to help distributed library networks and content.
A very significant technical problem for ADR was validation of incoming records to make sure the minimum MODS metadata was there and was correct. They decided that XSD could both the valid structure for XML encoded MODS records and the correct data types as they had agreed upon. For instance a record would have not only the correct structure of metadata that a DTD might confirm, but also the correct data inside the metadata fields (for example the correct time, date, price, or URI format).
The problem derived from the fact that the packaged XSDs as used by Fez were not applicable for ADR's purposes, which meant the creation of new XSDs. Integration of new custom XSDs with Fez was poorly documented and ADR encountered functional bugs with Fez. A survey of Fez use revealed that no one else was trying to significantly modify Fez's XSD templates, and ADR's number of digital objects, each with unique metadata structures, was very large. Added to this was the need to crosswalk MODS to DC that required a whole new set XSDs for the DC metadata records.
ADR ended up authoring their own "document types" (MODS and DC XSD-validated metadata records) for use. That process is long and complex and ultimately did not provide the best treatment for heterogeneous objects (objects containing multiple genres of document types, like a web page) since Fez allowed such minimal description for a single document type. This forced them to store these objects as separate records.
What interests me with this report is not so much the specific technical problems and solutions they encountered (which are kind of torturous to read about) but that such huge obstacles can be created when the technical tools are insufficient to the task. On the one hand, MODS was easily selected and implemented for a number reasons, not least of which is that Fez worked smoothly with it. There are just a few paragraphs on it. On the other hand, entire collections of XSDs and the creation of a new conceptual entity (document types) is created largely because Fez has a poor capacity to deal with new or modified XSDs. Four pages detail this struggle, and it's unresolved at the conclusion of the article. I conclude that while the biggest struggles for digital curation and interoperability may be social, political, cultural and so on, the extent to which technical can grease the wheels is tremendous.
This article reports the efforts Alliance Digital Repository (ADR), a consortial digital repository service for the Colorado Alliance of Research Libraries (Alliance), to create a standard descriptive metadata policy for their repository records. Members in the Alliance included twelve separate libraries at nine different academies and institutions, public and private. The goal was to support each library's community standards while also ensuring interoperability. This would facilitate a central repository.
Needless to say the scope for a project like this, which is charged with accommodating a dozen libraries each with their own practices and metadata (not to mention unique data both from within and without the library), expands at a rate equal to thought itself.
The ADR eventually pared down the libraries' metadata they had to consider to:
- MARC
- MARC XML
- MODS
- DC
- Various extensions to these standard schemas
- ProQuest's Digital Dissertations XML-based metadata schema
As they note Fedora does not have a user-friendly interface for submission and searching built-in (like DSpace) so they chose Fez as a configurable front end, adding that other interfaces could be attached to Fedora as needed.
MODS was chosen as the "normalizing schema," the one all records would be converted to at a minimum no matter what metadata scheme they had upon ingestion. The report gives a really helpful list of reasons why, and I think it's interesting that #1 is the simple fact that Fez and MODS "are proven partners: The Fez system was already using MODS widely throughout the system as the primary means of describing objects." They give other factors: the right granularity, crosswalks to the popular data-sharing DC, and plain familiarity, but it's notable that the tool in this case (Fez) partly determined yet another tool.
ADR determined minimal metadata field requirements with DLF Aquifier, a Digital Library Federation initiative to help distributed library networks and content.
A very significant technical problem for ADR was validation of incoming records to make sure the minimum MODS metadata was there and was correct. They decided that XSD could both the valid structure for XML encoded MODS records and the correct data types as they had agreed upon. For instance a record would have not only the correct structure of metadata that a DTD might confirm, but also the correct data inside the metadata fields (for example the correct time, date, price, or URI format).
The problem derived from the fact that the packaged XSDs as used by Fez were not applicable for ADR's purposes, which meant the creation of new XSDs. Integration of new custom XSDs with Fez was poorly documented and ADR encountered functional bugs with Fez. A survey of Fez use revealed that no one else was trying to significantly modify Fez's XSD templates, and ADR's number of digital objects, each with unique metadata structures, was very large. Added to this was the need to crosswalk MODS to DC that required a whole new set XSDs for the DC metadata records.
ADR ended up authoring their own "document types" (MODS and DC XSD-validated metadata records) for use. That process is long and complex and ultimately did not provide the best treatment for heterogeneous objects (objects containing multiple genres of document types, like a web page) since Fez allowed such minimal description for a single document type. This forced them to store these objects as separate records.
What interests me with this report is not so much the specific technical problems and solutions they encountered (which are kind of torturous to read about) but that such huge obstacles can be created when the technical tools are insufficient to the task. On the one hand, MODS was easily selected and implemented for a number reasons, not least of which is that Fez worked smoothly with it. There are just a few paragraphs on it. On the other hand, entire collections of XSDs and the creation of a new conceptual entity (document types) is created largely because Fez has a poor capacity to deal with new or modified XSDs. Four pages detail this struggle, and it's unresolved at the conclusion of the article. I conclude that while the biggest struggles for digital curation and interoperability may be social, political, cultural and so on, the extent to which technical can grease the wheels is tremendous.
Saturday, October 17, 2009
Cooliris
Cooliris, is a 3-D image wall that allows you to visually browse images, whether they be in Google Image, YouTube, Flickr, Facebook, Picassa, images in one's harddrive and recent versions of the photo database program Adobe Lightroom.
I was sad to discover that I could not download this program to my own Mac Mini as it features Mac OX 10.4, Power PC, and apparently I cannot upgrade to the necessary Leopard version that supports Cooliris. Update: Previous versions (tho unsupported) are available.
You can share your 3-D image wall through a url, as well as bookmark and save it. It allows one to "jump freely" through Flickr image pools and sets - though not sure how that differs from the current way we maneuver from image pools to sets - does "jump freely" = "move fluidly?" Not sure.
It claims usability with hundreds of sites due to its being built around Media RSS format. It advertises its ability to be used to view numerous television and movie episodes on Hulu, etc. Its slideshow feature allows one to double click and launch slide shows that one can pause and rewind. It claims to have the fastest way online to search images with its' style of "zipping" through the 3-D image wall.
Again this makes me think back to the Visible Archive from Australia - and its archival metadata visualization. If there was a way to integrate these technologies and allow one to browse visualizations of data - the future could be very exciting.
I was sad to discover that I could not download this program to my own Mac Mini as it features Mac OX 10.4, Power PC, and apparently I cannot upgrade to the necessary Leopard version that supports Cooliris. Update: Previous versions (tho unsupported) are available.
You can share your 3-D image wall through a url, as well as bookmark and save it. It allows one to "jump freely" through Flickr image pools and sets - though not sure how that differs from the current way we maneuver from image pools to sets - does "jump freely" = "move fluidly?" Not sure.
It claims usability with hundreds of sites due to its being built around Media RSS format. It advertises its ability to be used to view numerous television and movie episodes on Hulu, etc. Its slideshow feature allows one to double click and launch slide shows that one can pause and rewind. It claims to have the fastest way online to search images with its' style of "zipping" through the 3-D image wall.
Again this makes me think back to the Visible Archive from Australia - and its archival metadata visualization. If there was a way to integrate these technologies and allow one to browse visualizations of data - the future could be very exciting.
Wednesday, October 14, 2009
Dude, Your in the NYT
Happy off week everybody, here is a light piece but one which should still be interesting to us.
Congratulations, now when people ask you about what exactly it is that you are doing in school you don't have to give them vocab heavy monologues about metadata, data migration and petabytes, you can point them to this article in the New York Times.
The heart of this article is what we have been discussing through much of the semester, the sciences are going through a major change where the shear amount of data that is being gathered threatens to overwhelm researchers who don't have the training to deal with such datasets. The article goes then discusses how current students in the sciences are unable to access the vast scope of the information sets which are a part of their field s due to the limited nature of the computational facilities available to students. This causes the young minds to “imprint on these small systems, that becomes their frame of reference and what they’re always thinking about...”. I have doubts on the validity of this but still Google and IBM are going to solve these problems for scientist by providing them with access to better computers.
There is of course the unstated belief that shadows this article that all of these issues are inherently technical, and the application of computational power will solve the petabyte problem. Indeed the article quotes on researcher saying “Science these days has basically turned into a data-management problem”. A brief Google search does find that the scientist quoted here is a biology/ genetics researcher so in that field this might be a perfectly valid point, but I wonder if all scientist feel this way. And if they do why, and if they don't why not?
The question that I suppose I have from this article is where do we fit in? Do we work with the scientist to help them understand their data or are we only concerned with where the data is kept, and how to ensure that it remains available for use now and in the future?
I don't have good answers I have been looking at these problems for the better part of a year now and I am still not sure how we as a profession fit with this area, or if we do.
Congratulations, now when people ask you about what exactly it is that you are doing in school you don't have to give them vocab heavy monologues about metadata, data migration and petabytes, you can point them to this article in the New York Times.
The heart of this article is what we have been discussing through much of the semester, the sciences are going through a major change where the shear amount of data that is being gathered threatens to overwhelm researchers who don't have the training to deal with such datasets. The article goes then discusses how current students in the sciences are unable to access the vast scope of the information sets which are a part of their field s due to the limited nature of the computational facilities available to students. This causes the young minds to “imprint on these small systems, that becomes their frame of reference and what they’re always thinking about...”. I have doubts on the validity of this but still Google and IBM are going to solve these problems for scientist by providing them with access to better computers.
There is of course the unstated belief that shadows this article that all of these issues are inherently technical, and the application of computational power will solve the petabyte problem. Indeed the article quotes on researcher saying “Science these days has basically turned into a data-management problem”. A brief Google search does find that the scientist quoted here is a biology/ genetics researcher so in that field this might be a perfectly valid point, but I wonder if all scientist feel this way. And if they do why, and if they don't why not?
The question that I suppose I have from this article is where do we fit in? Do we work with the scientist to help them understand their data or are we only concerned with where the data is kept, and how to ensure that it remains available for use now and in the future?
I don't have good answers I have been looking at these problems for the better part of a year now and I am still not sure how we as a profession fit with this area, or if we do.
google scholar. not there yet.
Given how much class time has been devoted to discussing the many aspects of Google, I thought I would find this week’s article using Google Scholar. A quick search for the key term “data sharing” (all articles) revealed a top hit for a 1994 article titled “The Bayou architecture: Support for data sharing among mobile users” (http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=512726). Cited by 329 other sources, this article is a good 50 citations ahead of its closest competitor. Clicking through to the list of articles that cited the Bayou article in order of the article’s citation instances might be more useful if there were options as to how this list were arranged (say chronologically). Going back to the original search page and changing the search parameters to limit articles by date (say “since 2008”) still yields some 72,000 hits, however, in this function one can no longer see the returns based on number of citations.
In an article titled "Newswire Analysis: Google Scholar’s Ghost Authors, Lost Authors, and Other Problems," which appeared in Library Journal (9/24/2009), the author criticizes Google Scholar for other reasons: namely their poor metadata standards, and how this will effect citation numbers. He notes that because Google Scholar relies on their own algorithms rather than utilizing the metadata available through libraries. This, he argues “creates phantom authors for millions of papers. They derive false names from options listed on the search menu, such as P Login (for Please Login).”
It seems clear, that Google Scholar is a long way from having any sort of reliable measure of citation count, and also is not yet the powerful search tool it purports to be.
Blog Post About Nothing
So, I ran across this hypothetical Seinfeld scene mocking the twitter phenomenon, and couldn't resist turning it into a blog post. Basically, in the scene, Elaine and Jerry are giving George a hard time because he has become obsessed with tweeting and is engaging in said activity compulsively. George is staunchly defending his right to tweet (declaring "No one stops George Costanza from tweeting!") When Jerry points out that George, who has no job or girlfriend, does not in fact have anything of interest to tweet about, George emphatically asserts, "I got plenty to tweet about, baby!" After Kramer wanders in and mentions that he had a friend who tweeted himself to death, the scene ends with Jerry asking rhetorically whether typing in increments greater than 140 characters would break the internet.
The scene, though obviously farcical, illustrates several points relevant to the consideration of digital curation. First, it brings to mind the problem with noise potentially drowning out information of value. Digital publication is so cheap and easy that anyone, even a consummate loser like Costanza, can flood cyberspace with information about nothing. Because of the potential to lose relevant information in a sea of trivia, I think that it is extremely important that we emphasize the filter function involved in digital curation. For example, can any single scientist really humanly go through ALL data pertaining to a study? Particularly in fields like climatology it seems essential to perform some sort of data triage. (Many of those fields benefit from shared data, but usually also along the lines of collaborative efforts instead of straight data dumps.) It seems to me that digital curators could at least perform a cursory, primary triage of data.
A second point raised by the Seinfeld skit is that no matter how great and convenient we the information pros find new information technology, there will always be the likes of ludites, tenured liberal arts professors emeriti, and half-deranged hipsters who will hate, fear, or refuse to accept new technological methods. In expanding access via digital means, we must be careful not to limit access to other portions of the population, which may be more a problem of infrastructure than straight-up digital curation. At the very least, education programs in new i.t. methods seem necessary.
Finally, Jerry's joke about 141 characters breaking the internet drives home the fact that we really are dealing with new media. Twitter has changed the way people communicate by rigidly enforcing a strict brevity. This may change people's reading/learning habits. I'm not precisely suggesting we limit our descriptive elements to 140 characters per digital item, but it is something to keep in mind.
So, basically, the sketch brought to the surface of my mind some of the digital curation issues that have been spinning around in there such as filtering noise, adapting to new user habits, yada, yada, yada...
Curating Architectural 3D CAD Models
MacKenzie Smith's article presents MIT’s FACADE (Future-Proofing Architectural Computer-Aided DEsign) research project as an attempt to determine how to preserve and provide access to 3D CAD digital records. The digital curation project includes working with three architectural firms to determine their workflow, usage, and expectations about archival preservation of their digital design records. Traditionally, architectural archives have included records about the design of architectural projects, as well as firm records that support the creation of buildings. Numerous documents are generated through the process of architectural design and these records are now largely digital born – email correspondence, digital photography, 2D CAD renderings, and 3D CAD models, to name a few. Architects generally use proprietary software to create many of their design. Most of these files are easily migrated or converted to standard formats, such as PDF or JPEG. The problem arises when trying to preserve or access 3D architectural models, created using proprietary software in non-standard formats, which cannot be easily migrated or converted.
This presents a major problem for libraries, archives, and museums whose goals would include providing access to the digital records, but would not have the funding to support the ownership and maintenance of proprietary software systems, especially when the software used to create the documents rapidly changes. The FACADE project looked at strategies for exporting files and creating standard formats. This required expertise in both the CAD software and the underlying data, which many librarians or archivists at repositories would not have. The hope is that this process will become more automated over time.
One of the primary issues is how to connect the various records created within context of the design process. The 3D model is most valuable when viewed in relation to other drawings, photographs, and project correspondence. To this end, they have established a “Project Information Model” which will collect 3D model data and connect it to other project records. They have created a prototype for open source software that will assist curators in creating metadata.
The FACADE project web site includes further information, including a final report on the project. The final report outlines four versions in different formats that will be used to preserve the 3D CAD models.
1. Original (the originally submitted version of the CAD model)
2. Display (an easily viewable format to present to users, normally 3D PDF)
3. Standard (full representation in preservable standard format, normally IFC or STEP)
4. Dessicated (simple geometry in a preservable standard format, normally IGES)
This project was funded by a two-year grant from IMLS. The project predominantly set out to conduct research on the best methods and practices for curating 3D CAD models. There are still many research questions to be addressed, such as those pertaining to privacy and copyright of the data.
This presents a major problem for libraries, archives, and museums whose goals would include providing access to the digital records, but would not have the funding to support the ownership and maintenance of proprietary software systems, especially when the software used to create the documents rapidly changes. The FACADE project looked at strategies for exporting files and creating standard formats. This required expertise in both the CAD software and the underlying data, which many librarians or archivists at repositories would not have. The hope is that this process will become more automated over time.
One of the primary issues is how to connect the various records created within context of the design process. The 3D model is most valuable when viewed in relation to other drawings, photographs, and project correspondence. To this end, they have established a “Project Information Model” which will collect 3D model data and connect it to other project records. They have created a prototype for open source software that will assist curators in creating metadata.
The FACADE project web site includes further information, including a final report on the project. The final report outlines four versions in different formats that will be used to preserve the 3D CAD models.
1. Original (the originally submitted version of the CAD model)
2. Display (an easily viewable format to present to users, normally 3D PDF)
3. Standard (full representation in preservable standard format, normally IFC or STEP)
4. Dessicated (simple geometry in a preservable standard format, normally IGES)
This project was funded by a two-year grant from IMLS. The project predominantly set out to conduct research on the best methods and practices for curating 3D CAD models. There are still many research questions to be addressed, such as those pertaining to privacy and copyright of the data.
More on Google Books
Sergey Brin wrote an opinion article late last week for the New York Times titled "A Library to Last Forever." The same article appeared a day later as a blog post titled A Tale of 10,000,000 Books on the official Google blog. The post is essentially a defense of Google Books and the recent settlement they have reached with the Authors Guild and the Association of American Publishers.
In the post, Brin talks about a "black hole" of information printed after 1923. These books are not in the public domain, but are frequently out of print and increasingly difficult to find. Brin argues that Google Books is making important parts of the "world's collective knowledge and cultural heritage" available. I don't think any of us would argue the potential for the positive impact of Google Books on the way research is conducted. I am more concerned about what Google Books means for future mass digitization projects.
Brin states that "If Google Books is successful, others will follow." But what about the large startup costs? To Google's credit, they are the only ones with the means and the drive to even attempt such a large-scale book digitization project. But with the expense being so high, will there ever be anyone else with both the means and drive to create another digital collection? It seems to me that it would be highly unlikely, especially if Google Books is successful. Who wants to take on a giant when the cost of entrance is so high?
While I don't pretend to know everything about the settlement, it appears that it will create a registry of rights for books which helps to identify the rights holders for individual works and to make obtaining permission and assigning appropriate revenue to the rights holders easier. Again, such a registry would be very helpful, but how many entities exist with the kind of funding it would take to pay the appropriate fees and digitize the books?
And if Google Books is the only entity to attempt such a large-scale book digitization project, we find ourselves back where we started with the problems of inaccurate and misleading metadata. Brin addresses these issues briefly, only to say that Google is "working hard to address them."
In spite of all of my doubts, it is hard not to get swept up by Brin's call to save the literary works of the 20th century. I suppose we'll have to wait and see how Google Books affects the future of mass digitization projects. Meanwhile, Google would like us believe that they have found the answer.
In the post, Brin talks about a "black hole" of information printed after 1923. These books are not in the public domain, but are frequently out of print and increasingly difficult to find. Brin argues that Google Books is making important parts of the "world's collective knowledge and cultural heritage" available. I don't think any of us would argue the potential for the positive impact of Google Books on the way research is conducted. I am more concerned about what Google Books means for future mass digitization projects.
Brin states that "If Google Books is successful, others will follow." But what about the large startup costs? To Google's credit, they are the only ones with the means and the drive to even attempt such a large-scale book digitization project. But with the expense being so high, will there ever be anyone else with both the means and drive to create another digital collection? It seems to me that it would be highly unlikely, especially if Google Books is successful. Who wants to take on a giant when the cost of entrance is so high?
While I don't pretend to know everything about the settlement, it appears that it will create a registry of rights for books which helps to identify the rights holders for individual works and to make obtaining permission and assigning appropriate revenue to the rights holders easier. Again, such a registry would be very helpful, but how many entities exist with the kind of funding it would take to pay the appropriate fees and digitize the books?
And if Google Books is the only entity to attempt such a large-scale book digitization project, we find ourselves back where we started with the problems of inaccurate and misleading metadata. Brin addresses these issues briefly, only to say that Google is "working hard to address them."
In spite of all of my doubts, it is hard not to get swept up by Brin's call to save the literary works of the 20th century. I suppose we'll have to wait and see how Google Books affects the future of mass digitization projects. Meanwhile, Google would like us believe that they have found the answer.
Visualize This.
For the past few weeks I've been interested in how large data sets are being "visualized". Not only is this a fun (and easy) way to digest large issues, but the data must come from somewhere.
Some blogs I've been looking at lately include Information Aesthetics, Information is Beautiful, and Ben Fry's Projects, especially a visualization of changes in editions of Darwin's Origin of the Species.
I explored these sites to see where the authors are getting their data. Information is Beautiful provides links to his sources, which open as Google document spreadsheets. David McCandless, the author, also links to The Guardian's data (since the newspaper's DataBlog is also linked) and this also opens to spreadsheets. Information Aesthetics' Andrew Vande Moere doesn't provide his data sources, but does offer links to other data/information sites for hours of wasting/exploring. Ben Fry's The Preservation of Favoured Traces project states the "text for each edition was sourced from their careful transcription of Darwin's books" although more information about how the text was actually transcribed. The "Data" link on his site, however, is grayed out, so apparently not available. Fry does explain all his projects in detail, however, just maybe not all the information we would want.
An interesting article on the Visual Journalism blog talks about how some data is just boring and not everything needs to be graphically interpreted. The author, Gert K. Nielsen, is discussing a NY Times interactive visual about how people spend their day and mentions that the original data of the graphic, "American Time Use Survey, is in danger of being shut down due to serious cuts in the budget of the Bureau of Labor Statistics". That would hamper the continuation of this visualization, but the Bureau of Labor Statistics will surely have other great data to troll.
The popularity of information visualization will probably only increase (until the novelty runs out?). Ben Fry has written a book about it, Visualizing Data, published by O'Reilly. I wonder if the graphs and pretty colors will distract people from questioning the source of the data, or highlight the nature of statistics and data even more, creating a demand for source knowledge. I'm hoping the latter, especially with easy sources like data.gov, there is no reason not to provide the raw data. I also like the possibility of increased data sharing this might facilitate. Even if it starts out small, such as the GoogleDocs on Information is Beautiful, this creates a society of openness that can be emulated.
Galaxy Zoo 2
These past couple of weeks, I’ve been blogging about developing cyber initiatives and grants being given to fund projects that designed to generate and display data like never before. In fact, the words “revolutionary data collection” kept popping up again and again. But in most of the articles I’ve been reading about developing projects, no one ever really delves into what, exactly, will be done with the massive amount of data that will be collected; no one ever mentions how the data could be used – just that it will be utilized in “revolutionary” ways. With this in mind, this week I set out to find an example of something revolutionary – or at the very least something cool and interesting – that was being done with digital data.
My search led me to the zoo, the Galaxy Zoo, that is. Galaxy Zoo is a site that invites the public to help classify millions of galaxies using data and images from the Sloan Digital Sky Survey at the Apache Point Observatory in New Mexico. All of the images come from a digital camera which is mounted on a telescope.
I think this is really neat. People like you and me can help classify and study the universe, no prior knowledge of astronomy needed (or so claims Wikipedia). The original Galaxy Zoo first made its debut in 2007 “with a data set made up of a million galaxies imaged with the robotic telescope of the Sloan Digital Sky Survey”; the new version, Galaxy Zoo 2, launched back in February of this year. According to their website, within the first day of Galaxy Zoo’s launch the site was getting 70,000 classifications an hour; during the first year over 50 million classifications were received from nearly 150,000 people.
Galaxy Zoo 2 has focused on almost a quarter of a million of the “nearest, brightest and most beautiful galaxies” for users to view and classify. Apparently, new discoveries by the amateur astronomers at Galaxy Zoo are being made constantly about galaxy colors, shapes and patterns. Professional telescopes (and I’m assuming professional astronomers along with them) have even followed up on some of the Galaxy Zoo findings; the list of professionals includes “the Isaac Newton and William Herschel Telescopes on the island of La Palma in the Canaries, Gemini South in Chile, the WIYN telescope on Kitt Peak, Arizona, the IRAM radio telescope in Spain’s Sierra Nevada, the Swift and GALEX satellites, and the Hubble Space Telescope.”
I've been playing around with the website's classification tutorial page, and I have to say that it is pretty fun. This is a great example of the neat things that can be accomplished with crowd-sourcing and open data. Regular people with an interest in astronomy having access to data to classify galaxies, make discoveries, and help figure things out about the universe; Galaxy Zoo 2 may not be mind-blowingly revolutionary but it definitely fit into the cool and interesting category.
My search led me to the zoo, the Galaxy Zoo, that is. Galaxy Zoo is a site that invites the public to help classify millions of galaxies using data and images from the Sloan Digital Sky Survey at the Apache Point Observatory in New Mexico. All of the images come from a digital camera which is mounted on a telescope.
I think this is really neat. People like you and me can help classify and study the universe, no prior knowledge of astronomy needed (or so claims Wikipedia). The original Galaxy Zoo first made its debut in 2007 “with a data set made up of a million galaxies imaged with the robotic telescope of the Sloan Digital Sky Survey”; the new version, Galaxy Zoo 2, launched back in February of this year. According to their website, within the first day of Galaxy Zoo’s launch the site was getting 70,000 classifications an hour; during the first year over 50 million classifications were received from nearly 150,000 people.
Galaxy Zoo 2 has focused on almost a quarter of a million of the “nearest, brightest and most beautiful galaxies” for users to view and classify. Apparently, new discoveries by the amateur astronomers at Galaxy Zoo are being made constantly about galaxy colors, shapes and patterns. Professional telescopes (and I’m assuming professional astronomers along with them) have even followed up on some of the Galaxy Zoo findings; the list of professionals includes “the Isaac Newton and William Herschel Telescopes on the island of La Palma in the Canaries, Gemini South in Chile, the WIYN telescope on Kitt Peak, Arizona, the IRAM radio telescope in Spain’s Sierra Nevada, the Swift and GALEX satellites, and the Hubble Space Telescope.”
I've been playing around with the website's classification tutorial page, and I have to say that it is pretty fun. This is a great example of the neat things that can be accomplished with crowd-sourcing and open data. Regular people with an interest in astronomy having access to data to classify galaxies, make discoveries, and help figure things out about the universe; Galaxy Zoo 2 may not be mind-blowingly revolutionary but it definitely fit into the cool and interesting category.
Wednesday, October 7, 2009
Google ambitions ignore digital ephemera of yesteryear
Wired recently had an article critiquing Googles ambitious reach into digital curation of books with a reminder of the last large digital library it undertook and all but abandoned. Usenet.
First Google rescued post-1995 Usenet (a dial-up message board system founded in 1980) from Dejanews in 2001. Google later was able to add millions of posts to this from the magtape of an old Unix guru, Marc Spencer. This gave Google a digital library from over 2 decades of 700 million articles from 35,000 newsgroups.
Wired interviews the Unix guru whose archive supplies a bulk of Google groups (and can't seem to get his name right - is it Marc or Henry Spencer?). Speaking on behalf of his community of Usenet users, Mr. Spencer is very disappointed with the poor searchibility of the material he provided Google.
I would ask: Where are the finding aids? How are things catalogued? Does Google use any of the methods that have worked for traditional archives spanning hundreds of years? No, Google can't even retrieve the legendary alt.gothic flamewars of 1993.
Well, perhaps the world is a better place for that.
But in all seriousness, if anything required professional curation, Usenet could be that. Many from my generation compiled subject knowledge in the early 90s in the form of F.A.Qs, discographies, videographies, and many other encyclopediac collections of unpublished popular (or not very popular) material. Did we save it all? Probably not. There were news-groups for great swaths of information that probably did not carry over to the world wide web post 1995, or even post 2001.
Granted, an enormous amount of Usenet one would not want to have see the light of day, for reasons legal and sundry that could make Craigslist pale in comparison. Still, there are volumes of music, film, and sub-cultural anthropology I have in my file cabinets printed on dot-matrix printers in 1992 that I have yet to see mirrored online or in zines, magazines or books. As I get older and de-clutter, and deem certain material unnecessary or not keeping in with my current interests or values, that information will most likely be lost.
I think a great idea would be for there to be a way to feed and properly code Usenet material (i.e. Usenet group "such and such" date: 1996) and feed the archive anonymously into Wikipedia.
I think the anonymous Wiki framework would be the best platform to mine and add Usenet knowledge and curate it in a central way that would be searchable. Usenet entries would be date-logged as historical data, and contestation, edits and updates would need to follow the historical Usenet information. This would give Wiki a valuable historical layer that I currently find missing. There are times when Wiki feels to me like urban Las Vegas, where the old is torn down and forgotten to make room for the new. What is Wiki kept historical layers to their entries? I think it would be interesting to see a Wiki article from 7 years ago on a given topic.
First Google rescued post-1995 Usenet (a dial-up message board system founded in 1980) from Dejanews in 2001. Google later was able to add millions of posts to this from the magtape of an old Unix guru, Marc Spencer. This gave Google a digital library from over 2 decades of 700 million articles from 35,000 newsgroups.
Wired interviews the Unix guru whose archive supplies a bulk of Google groups (and can't seem to get his name right - is it Marc or Henry Spencer?). Speaking on behalf of his community of Usenet users, Mr. Spencer is very disappointed with the poor searchibility of the material he provided Google.
I would ask: Where are the finding aids? How are things catalogued? Does Google use any of the methods that have worked for traditional archives spanning hundreds of years? No, Google can't even retrieve the legendary alt.gothic flamewars of 1993.
Well, perhaps the world is a better place for that.
But in all seriousness, if anything required professional curation, Usenet could be that. Many from my generation compiled subject knowledge in the early 90s in the form of F.A.Qs, discographies, videographies, and many other encyclopediac collections of unpublished popular (or not very popular) material. Did we save it all? Probably not. There were news-groups for great swaths of information that probably did not carry over to the world wide web post 1995, or even post 2001.
Granted, an enormous amount of Usenet one would not want to have see the light of day, for reasons legal and sundry that could make Craigslist pale in comparison. Still, there are volumes of music, film, and sub-cultural anthropology I have in my file cabinets printed on dot-matrix printers in 1992 that I have yet to see mirrored online or in zines, magazines or books. As I get older and de-clutter, and deem certain material unnecessary or not keeping in with my current interests or values, that information will most likely be lost.
I think a great idea would be for there to be a way to feed and properly code Usenet material (i.e. Usenet group "such and such" date: 1996) and feed the archive anonymously into Wikipedia.
I think the anonymous Wiki framework would be the best platform to mine and add Usenet knowledge and curate it in a central way that would be searchable. Usenet entries would be date-logged as historical data, and contestation, edits and updates would need to follow the historical Usenet information. This would give Wiki a valuable historical layer that I currently find missing. There are times when Wiki feels to me like urban Las Vegas, where the old is torn down and forgotten to make room for the new. What is Wiki kept historical layers to their entries? I think it would be interesting to see a Wiki article from 7 years ago on a given topic.
Subscribe to:
Posts (Atom)