Group

Group Admins

  • Profile picture of Adela Sobotkova

Digital Data Collection

Public Group active 1 year, 2 months ago

Welcome to the Digital Data Collection group, which focuses on the topic of digital data collection in the field, its benefits and challenges. The group was started by the FAIMS team, which is based at Macquarie University in Sydney, Australia. We are happy to discuss our own open source data collection software (FAIMS Mobile Platform) as well as other mobile technologies currently in use. If you are considering switching to a digital workflow in the field, or have experience with a particular software, please post your questions and share your experiences here!

FAIMS is not a walled garden : A Response to Ben

May 24, 2016 in uncategorized by Adela Sobotkova

Dear Ben and fellow DAC members,

we at FAIMS have not been active on the blog, because we were working on a major grant application and overdue publications. We are now returning into the game, because we want to get the community feedback on the next upgrade of FAIMS, and because we were inspired by Ben Carter’s post.

We will try to respond to several points that Ben Carter made in a series of posts .  This first one will focus on a misunderstanding of FAIMS evident in Ben Carter’s statement of : “I don’t want to create a single suite of tools that must be used together- that can be limiting as well (if you want this, I recommend you check out FAIMS. Flexibility is the name of the game.”

Ben suggests that FAIMS project offers a single suite of tools that is like a walled garden, where you either buy into the whole suite or nothing at all. The opposite is the reality. FAIMS has been and is building discreet tools for specific field research data tasks, that work together with other data services. FAIMS users are free to pick whatever suits their needs. FAIMS makes it possible, but does not mandate, that archaeologists use an entire data lifecycle. And so, you can use FAIMS Mobile to collect your data and either export it to your desktop software for further processing, or export it to Open Context or tDAR or your favorite MS Access db or other regional register for publication, sharing, or follow up work. None of these steps are required in any way, FAIMS project just makes them technically possible through the FAIMS ecosystem.

The FAIMS Mobile platform for offline data collection (the flagship product of the FAIMS project and one part of the ecosystem) is entirely customizable. Imagine a bucket of functionality that you can pick from: want spatial data? Get a mapping screen and figure out your projection. Need dropdowns or picture dictionaries? Pick those and populate them with illustrating images. Don’t care about recording video? Drop it. The FAIMS Mobile is flexible: it can do as little or as much as anyone needs.

The FAIMS ecosystem is in my opinion the very “concatenation of tools” that Ben calls for. We call it a federated system because that’s what it is – a federation of tools managed by FAIMS and FAIMS partners (Open Context, Digital Antiquity), who are committed to the best practices and digital data standards, open source code, reproducibility and sustainability, etc.., – but which also lets archaeologists opt out and use their preferred desktop software at any time.

Later in the post, Ben proposes that archaeologists should use a palette of tools, which have been developed for a particular workflow. You can use survey forms for structured data, or QGIS for spatial data collection and visualisation, and so on. Yet, if your projects include multiple workflows, such as you need to take pictures while filling a form, than you are faced with the management of multiple disparate data sources and pieces of software,  and need to ensure their integration, completeness and consistency at the end of your fieldwork. Do all records have associated images, do all images have the right labels and associations? If you work at a multi-year project, you also need to monitor the status of each piece of software, its compatibility with your hardware, latest upgrades, connectivity to external sensors etc., and constantly evaluate if they are still suitable for your task at hand. We see this as a major overhead.

The advantage of FAIMS over this palette of separate tools is that if you use FAIMS Mobile you do not need a separate application for pictures, a separate application for GIS data, a separate application for diaries, or structured data. FAIMS Mobile creates all of these datatypes together as part of a unified record. On export you have the choice of either exporting all the data as an integrated whole (sqlite database) or in discrete groups of data types (csvs, pictures, GIS shapefiles or kmls). If you choose the latter, all exported data groups will have the attributes of the record embedded in it (picture will have data embedded in it, GPS point shapefile will have an attribute table, etc.), so that if your pictures get separated, they can always be identified and traced back to their origin. And tedious, labor-intensive tasks, such as the labeling of images, will be done for you by the computer, automatically, in one push of a button.
FAIMS has other features that are designed for field researchers, which we will discuss later. The bottom line here is that archaeologists should not need to compromise using software that was designed for other domains. Neither should should they need to manage and maintain several complex pieces of software while undertaking fieldwork. Some projects and small scale jobs may only require survey forms or GPS points, and will be well served by Excel or Garmin DNS. Yet, large scale projects that require integrated data collection and do not want to juggle multiple pieces of software while in the field can find many of their needs fulfilled by FAIMS.

New Version of a “Robust, Open, Flexible and Offline Digital Data Collection in the Field”

January 29, 2016 in uncategorized by Ben Carter

I am so excited that the Digital Archaeology Commons is up and running! Here’s my first contribution.

Much has changed since I first posted on the website of the Institute for Digital Archaeology back in September. That post, entitled, “ Robust, Open, Flexible and Offline Digital Data Collection in the Field” outlined my background and interest in digital data collection. I won’t reiterate that here. However, I will restate the goal and the basic tenets of my project. The goal is to collect a set of tools for archaeological data collection, quality assurance, storage and analysis that are open source, simple to use and “play well” with each other. I would like this project to result in a concatenation of tools that:
1. Is browser-based (and, therefore, device independent)
2. Can be used off-line (now possible with HTML5)
3. Employs open source code
4. Can be connected to a database of choice (proprietary or open-source)
5. Is inexpensive and simple (at least on the user end once it has been built)
6. Promotes the open use of resulting data

I reiterate here my dedication to these six tenets, but will stress that number 5 has inched its way up the ladder to position #1 (though it was never really intended as a hierarchical list). The ability for archaeologists (and others) who do not consider themselves proficient in digital tools to be able to “pick up” these tools and use them will relatively little guidance and exploit their value in the field has become the prime directive of this project. I keep thinking about those times in my life when money and time were sparse, such as when I was a graduate student (and new father), an adjunct and a visiting professor. In these stages of my life, I needed something cheap (okay, FREE), easy to use and powerful.

While I originally proposed to employ a spectacular tool known as wq.io, I became frustrated with the complexity of the tool and the long slog that it would likely take to get it up and running specifically for archaeology. Please know that I don’t consider this a problem with the tool, but a void in my own skill set- and yet that same void exists for many archaeologists. I forced myself to learn some programming- I knew very little. Indeed, leading up to the Institute last August, we were required to learn a number of web languages. What I learned for this activity is that I am a poor programmer. I can feel my shoulders tense just thinking about it! So, while I will need to do some data wrangling and, hence some programming, I want to minimize this. The goal is to create documentation and perhaps short lines of code (gonna need help on the latter) that can combine a wide variety of tools.

I also have learned that I neglected one central aspect to my requirements:
7. Must promote and enable the use of geospatial data and tools

This was always a requirement; after all archaeologists are spatial creatures. However, I realized that this was a de facto requirement; this addition, therefore, is simply an admission. Although the central purpose of the different components may not be handle geospatial data, they need to be designed with spatiality in mind.

Before I introduce my proposed tool set, it is incredibly important to note that nearly all of these tools have competitors both in the open software and in the proprietary software worlds. Indeed, in picking these tools I have tried to choose options that seem the most flexible. That way, if you are a dedicated user of Filemaker Pro or Microsoft Access, you can still use your preferred database management software with the other tools. I don’t want to create a single suite of tools that must be used together- that can be limiting as well (if you want this, I recommend you check out FAIMS. Flexibility is the name of the game.

So, here’s my proposed basic tool set along with some open and proprietary alternatives (this list is not exhaustive and many of these tools have different capabilities).

Form creation and data collection
Tool: KoboToolbox
Open source alternatives: ODK; Enketo Smart Paper; Ona
Proprietary alternatives: Fulcrum

Database
Tool: PostgreSQL with PostGIS
Open source alternatives: MySQL; MongoDB; Cassandra; SQLite with SpatialLite
Proprietary alternatives: Native formats for ArcGIS, MS Access or Filemaker Pro (see below); Oracle; Microsoft SQL; and many, many more.

Relational database management software
Tool: LibreOffice Base
Open source alternatives: Many, many options here. Pgadmin is the standard for PostgreSQL
Proprietary alternatives: Microsoft Access; Filemaker Pro

GIS
Tool:QGIS
Open source alternatives: gvSIG; uDig
Proprietary alternatives: ArcGIS; Global Mapper

I will leave off here. In future posts I will address the each individual component, their advantages and drawbacks and why each was eventually chosen. The goal, however, is not to try to justify the decision or convince an audience that this is the correct choice, but to engage in an open dialog about these different tools. I hope that others out there have their favorites and will contribute their perspectives. I look forward to hearing from you all!

Hello world!

December 23, 2015 in uncategorized by Adela Sobotkova

Welcome to Digital Archaeology Commons Sites. This is your first post. Edit or delete it, then start blogging!

Blog Activity

Close

Account Activated

Your account was activated successfully! You can now log in with the username and password you provided when you signed up.

Close

Account sign-in

Please use the form below to sign-in to your account.

Forgot password?
Close

Recover password

Please enter your username or email address. You will receive a link to create a new password via email.

We've sent you an activation link. Please check your inbox.

Close

Account signup

1 Account Info

2 Personal Info

Registering for this site is easy, just fill in the fields below and we'll get a new account set up for you in no time.

In order to avoid spam, automatic account registration is restricted to emails from the following domains – .edu, .org, .gov. To register with a different email address, please write to digitalarchaeology@matrix.msu.edu to request an account.

Success!

A confirmation link has been emailed to you

Skip to toolbar