Forum Replies Created

Viewing 15 posts - 1 through 15 (of 34 total)
  • Author
    Posts
  • #811

    Eric Kansa
    Participant

    Hi all, I direct Open Context (http://opencontext.org) and collaborate with the Digital Index of North American Archaeology (DINAA) project (http://ux.opencontext.org/archaeology-site-data/).

    I’m interested in meeting and collaborating with folks creating, curating and using State Government data.

    #803

    Eric Kansa
    Participant

    Hi Alice Lynn,

    Sounds great. Once you’ve put your data through Refine and feel like it is clean and consistent, please do send it on to us. We can provide another outside editorial eye, and then start working with you on peer review. It seems like your content will be in multiple places, so we’ll want to cross-reference so that we link all the different places on the Web that host your material.

    Looking forward to catching up in Lansing!
    -Eric

    #800

    Eric Kansa
    Participant

    Hi Ann,

    I’d tend to shy away from a license like CC BY-NC-SA. That is a very restrictive license, and does not qualify as “open data”. It would mean virtually no reuse, especially since nobody knows what “commercial” really means. I would be ok with that for indigenous materials, since the ethics would mean stakeholder concerns should come first. But without that kind of circumstance, I strongly recommend against it. In general, the most restrictive I’d go would be a CC-By-SA license. But us there a particular reason why you may want a restrictive license?

    Anyway, this is an issue that has been long discussed among people working on open acces and open data issues. Here’s a longer paper about it:
    http://alexandriaarchive.org/blog/wp-content/uploads/2012/Kansa-Open-Archaeology-Self-Archive-Draft.pdf

    and here’s a blog post:
    Creative Commons Non-commercial A Cruel Joke.

    here’s a long policy document about databases (for science) and licensing:
    https://wiki.creativecommons.org/wiki/Data

    Best!

    #762

    Eric Kansa
    Participant

    Hi All,

    To get you all started, I have a simple (ugly) demo of API search functionality here:

    http://ekansa.github.io/simple.html

    It’s just a start, but the example should be really useful for you all to adapt. You can test it with a search for “glass” or “bottle” to see that it works.

    Good luck!
    -Eric

    • This reply was modified 5 years, 5 months ago by  Eric Kansa.
    #760

    Eric Kansa
    Participant

    Sara + Nancy,

    One more thing, here’s the JSON result with the thumbnails included. I forget to specify that they needed to be specifically requested with the parameter “attributes=thumbnail”:

    http://opencontext.org/subjects-search/.json?prop=oc-gen-cat-object&proj=91-historic-fort-snelling&response=metadata,uri-meta&attributes=thumbnail

    #759

    Eric Kansa
    Participant

    Nancy, my examples about the Ecuador pottery apply just as much to you. The only difference is that the URL for your materials has parameters with different values that correspond to your specific dataset.

    To start, here’s an HTML view of objects in your data:
    http://opencontext.org/subjects-search/?prop=oc-gen-cat-object&proj=91-historic-fort-snelling

    And here’s a link to the (simple) JSON representation of you data:
    http://opencontext.org/subjects-search/.json?prop=oc-gen-cat-object&proj=91-historic-fort-snelling&response=metadata,uri-meta

    Last, here’s a link to a keyword search for “doll” in your objects (parameter: q=doll):
    http://opencontext.org/subjects-search/.json?prop=oc-gen-cat-object&proj=91-historic-fort-snelling&response=metadata,uri-meta&q=doll

    #758

    Eric Kansa
    Participant

    (1) On the downloadable data side, it would probably be easiest if Open Context generated a CSV dump of the data. I will be happy to prepare that dump once the data are in a final state (no longer actively edited). One can just link to the CSV data download.

    (2) For pictures, there are links to thumbnail images in the JSON search results (if the records have any associated pictures). The same JSON API also works for image searches in Open Context, where you can get thumbnail, preview-size, and full-size images.

    #757

    Eric Kansa
    Participant

    For searching, you’ll probably want to use the simple version of the API to get simple JSON results. JSON is easy to work with in Javascript to display as you like.

    For instance, if you start with an HTML search of your pottery, here’s the link:
    http://opencontext.org/subjects-search/Ecuador/Buen+Suceso?prop=oc-gen-cat-object—oc-gen-cat-pottery&proj=90-virtual-valdivia

    You can change this to a (simple) JSON response as so:
    http://opencontext.org/subjects-search/Ecuador/Buen+Suceso.json?prop=oc-gen-cat-object—oc-gen-cat-pottery&proj=90-virtual-valdivia&response=metadata,uri-meta

    Different parts of the URL are called “parameters”. These function to filter, sort, and ask for different kinds of data in Open Context’s responses. For Open Context, the parameter “q” is used for a keyword text searches. In the case above, you can add a parameter “q=incised” to add additional filter with a keyword search to the pottery from this project as so:
    http://opencontext.org/subjects-search/Ecuador/Buen+Suceso.json?prop=oc-gen-cat-object—oc-gen-cat-pottery&proj=90-virtual-valdivia&response=metadata,uri-meta&q=incised

    When you put this API / search functionality on your Web page, you’ll need to use Javascript. Javascript lets you program queries on the APIs of other websites (like Open Context) so you can do something with the data (like display the results). You’ll need to use some open source Javascript code to make it easy to get data from APIs. Since we’re in deep geekdom, I should let you know that getting data from APIs with Javascript usually involves “AJAX”(https://en.wikipedia.org/wiki/Ajax_(programming)) interactions. A common library to do this is called jQuery (https://jquery.com/).

    I have some Javascript code here:
    https://github.com/ekansa/ekansa.github.io

    You’re welcome to reuse anything. To make it easier for you to do your project, I’m starting a very simple bit of Javascript that will get data from the Open Context API so you can work with the results. It is not done, and I don’t have a demo for it yet, but the inprep javascript file is here: https://github.com/ekansa/ekansa.github.io/blob/master/javascript/opencontext-simple.js

    #756

    Eric Kansa
    Participant

    <p>One request was to have a search function to query Open Context and display results. This is well supported, but you need to understand something about how Open Context’s search API works. There are two main types of results for each search query:</p>

    1. Facets: Facets summarize metadata applicable to the entire set of results. For instance, if you do a search and the results come back from two different contexts, you’ll get facet results that count the number of records that exists in these two different contexts.
    2. Records: While facets provided summarized metadata about the entire set of results, records list individual records with some metadata that describe these records
    #727

    Eric Kansa
    Participant

    I have no idea how your data are structured and what your tables are. So not knowing your schema, it’s hard to respond to this. But, there are often many, many ways to relate data using SQL. Besides JOINs there are sub-queries (see: http://www.postgresqltutorial.com/postgresql-subquery/). Some of these approaches work differently on different databases, so fiddling and experimenting seems unavoidable.

    Sorry to not be very helpful! It’s super hard for me as an outsider to give more specific advice on this issue.

    #726

    Eric Kansa
    Participant

    What Dan said… 🙂

    Anyway, probably the best thing to do is look at the examples that illustrate how CSV plugins for Leaflet work in practice. Normally the people that write the plugins have some demo pages, and those are usually the best places to start in figuring out how to use the plugin.

    That said, there are lots of different programming styles in Javascript, so it can be really hard to grok, especially at first. I’m very easily confused by it still.

    #725

    Eric Kansa
    Participant

    Cool!

    A few notes of response:

    (1) Yes the Turkish diacritical marks are OK in Open Context (any UTF-8 characters work), the only issue I’ve seen is getting Excel to read the UTF-8 correctly with CSV dumps from Open Context. Libre Office handles it just fine, but Excel chokes on them.

    (2) Not all of your fields will map cleanly to Dublin Core. That’s OK!! You’ve got some fields that work for Dublin Core. For instance the “Cited In” field can be related to the Dublin Core terms “is referenced by” or “references”.  The PeriodO periods would probably relate to Dublin Core “temporal”.  Other fields are a bit more custom to your dataset.

    (3) It would be good to add a column to provide a URI for each one of the places in (location / valley / sites) in your  dataset. Geonames and Pleiades will be good to reference. When publishing with Open Context, we’ll make sure those places reference the gazetteer URIs you provide.

    (4)  On the issue of naming architectural features, you’re probably going to have to make up some identifier system to keep different ceiling crosses distinct.  You could be arbitrary like “Ceiling Cross 1”, “Ceiling Cross 2”. Or it could have some meaning “Ceiling Cross Northwest”. Or you can make it generic, and say “Architecture Feature 1”, “Architecture Feature 2”, etc. Then you can prove a descriptive attribute for “Architecture Feature 1” to call it a “Ceiling Cross”.  Most of the data we publish tends to use these sorts of generic identifiers (“Bone 1121”, “Sherd 23”, etc.), so I think that would work fine. Just to reduce chance for confusion, make sure your identifiers are unique for the whole scope of the project, so you don’t have multiple “Ceiling Cross 1” or “Architecture Feature 1” labels that actually refer to different real-world features.

    (5) On the category issue, it would be excellent for you to provide a definition for each category. Open Context would publish your definition, and each one of your category concepts / terms would have it’s own URI where that definition would exist. If some of the terms link up with the Getty Art and Architecture Thesaurus (AAT), that would be great to note also. Open Context can note equivalences to the AAT or note that maybe one of your terms is a more specific kind of classification to some more general category in the AAT.

    I hope this help!

     

     

     

    #697

    Eric Kansa
    Participant

    Hi Nancy,

    <p>I’ve reviewed your dataset and it looks great! I added a few URIs for Encyclopedia of Life taxa and UBERON for the mandibles, and I also expanded out some abbreviations (AAT, etc.). So I’m more or less ready to import.</p>

    <p>However I wanted to check with you about the spatial + stratigraphic organization of the data. I don’t see anything that looks like a clear context identifier, but I see there are some fields that seem related:</p>

    • HPU Number
    • Horizontal Coordinate 1
    • Horizontal Coordinate 2

    <p>I’d like to have some way of organizing the finds into context identifiers so we can have a hierarchy going from like:</p>

    <p>Ft. Snelling -> Ft. Snelling Barracks -> Context ID -> Object ID</p>

    <p>We can have multiple hierarchies, but one needs to be designated as the main one for easy navigation.</p>

    <p>Should I use “HPU numbers”? There seem to be lots of HPU numbers that are pretty descriptive, and also this field has lots of blank values. Or should I make new IDs based on the two fields for horizontal coordinates?</p>

    <p>We may also want to think about providing geospatial polygons (in GeoJSON format) for the different contexts. Then you could do some interesting mapping of the spatial distribution of objects within the barracks?</p>

    <p>Let me know!</p>
    <p>-Eric</p>

    • This reply was modified 5 years, 8 months ago by  Eric Kansa.
    #696

    Eric Kansa
    Participant

    Great! OK. When you a ready with the images, it’s probably easiest to make a table with a manifest of image file names, identifiers for the objects they describe, and any metadata you may have on the images (photographer, title, orientation, type of image, etc.). Dropbox is probably the easiest way to go.

    I should have time tomorrow to prepare your dataset your review in Open Context.

    #685

    Eric Kansa
    Participant

    Hi Nancy,

    I finally had a chance to look at the revisions in detail. They look really good to me, I especially appreciate how you updated the date fields to be internationally unambiguous. I see a few minor issues (for example I should probably normalize “surface” and “Surface” in the Vertical Reference Point field.), but these are easy for me to fix, since they don’t look conceptually hard. I can do that and push the results to GitHub and you can accept the “pull request” just so we’re all super hipster.</span>

    Do you want me to proceed with these minor clean ups and then import the data so you can review how they look in Open Context? Also, do you have image files? If so, we’ll need to arrange transfer. They can be hosted anywhere with persistence (Kora?) or we can arrange archiving with the CDL.

    Best, and awesome work with the Getty vocabularies!

    -Eric

     

    • This reply was modified 5 years, 8 months ago by  Eric Kansa.
Viewing 15 posts - 1 through 15 (of 34 total)