A framework for the publication of scientific databases

Today, the publication of research data plays an important role in astronomy and astrophysics. On the one hand, dedicated programs, such as surveys like SDSS and RAVE, data intensive instruments like LOFAR, or massive simulations like Millennium and MultiDark are initially planned to release their data for the community. On the other hand, for more traditionally oriented research projects data publication becomes a key requirement demanded by the funding agencies.

The common approach is to publish this data via dedicated web sites. This includes rather simple HTML forms as well as complex query systems such as SDSS-CAS or the RAVE database query interface. Most of these web sites are tailor made for the particular case and are therefore not easily transferrable to future projects.

At AIP, we gained experience with both the maintenance and the development of such applications. The RAVE and MultiDark databases, the German SDSS mirror, and several smaller web sites are written or at least maintained by us. It became, however, apparent that already the current plethora of applications constitutes a major challenge for maintenance expenses and scalability.

In order to address these issues, we are developing a new web framework, which is particularly designed to allow for different highly customizable web applications based on a common easily maintainable code base. It is called Daiquiri. As a service to the community the software is open source. Download Daiquiri and this Daiquiri application as a template / starting point.

Daiquiri is open source software and licensed under AGPL v3.

Download Daiquiri and Daiquiri-app at github!


Here are some features that come with Daiquiri. One of the key design goals of Daiquiri is, to each advanced complex feature to have a simple version that works out of the box. Many of the advanced features require 3rd party tools and are thus more complicated to set up. Further we strive to make Daiquiri as Virtual Observatory (VO) compliant as possible.

  • User management and user registration workflow
  • Contact messages to the support staff
  • SQL query interface with database browser and example queries
  • Customizable data query forms
  • SQL permission handling
  • Query management using MySQL Job Queue
  • Integration with PaQu, a parallel query engine
  • Database table viewer
  • First impression in-browser plotting
  • Viewer for linke images
  • Synchronous and asynchronous download into different formats (MySQL, CSV, VOTable)
  • Database table and column meta data management (supporting UCDs)
  • File links and downloads through database tables
  • UWS job submission
  • WordPress integration
  • Management of collaboration meetings

Daiquiri uses PHP together with the Zend framework. It is currently build for MySQL, but should be able to run with any database after modifications. The advanced features though are tightly integrated with MySQL extensions that we built for use with Daiquiri. These are the MySQL query queue, PaQu, and the IVOA MySQL VOTable dumper. The asynchronous data export relies on Gearman. The user interface makes use of jQuery, AngularJS, bootstrap, CodeMirror and flot.

Sites that run Daiquiri