BigQuery

BigQuery

Web address
Type of site	Infrastructure as a service
Registration	Required
Available in	English
Owner	Google
Launched	May 19, 2010
Current status	Active

BigQuery is a RESTful web service that enables interactive analysis of massively large datasets working in conjunction with Google Storage. It is an Infrastructure as a Service (IaaS) that may be used complementarily with MapReduce.

Design

BigQuery (BQ) is reportedly based on Dremel,^[1] a scalable, interactive ad hoc query system for analysis of read-only nested data. To use the data in BigQuery, it first must be uploaded to Google Storage and in a second step imported using the BigQuery HTTP API. BigQuery requires all requests to be authenticated, supporting a number of Google-proprietary mechanisms as well as OAuth.

Features

• Managing data - create and delete tables based on a JSON-encoded schema, import data encoded as CSV or JSON from Google Storage.
• Query - the queries are expressed in a SQL dialect and the results are returned in JSON with a maximum reply length of approximately 64 MB.^[2] There are some limitations to the usual SQL queries. For example, BigQuery supports joins, but one of the two JOINed tables must be small enough or use the JOIN EACH keyword instead.
• Integration - BigQuery can be used from Google Apps Script, Google Spreadsheets, or any language that can work with its REST API.
• Access control - is possible to share datasets with arbitrary individuals, groups, or the world.

Notes

1. Sergey Melnik, Andrey Gubarev, Jing Jing Long, Geoffrey Romer, Shiva Shivakumar, Matt Tolton, Theo Vassilakis (2010). "Dremel: Interactive Analysis of Web-Scale Datasets". Proc. of the 36th International Conference on Very Large Data Bases (VLDB).
2. "Google BigQuery API Overview (V2)". Retrieved 1 July 2012.

References

• "Google BigQuery - Getting Started". 2010.
• "Google BigQuery - Creating, Populating, and Deleting Tables". 2010.
• "Google BigQuery - Libraries and Scripts". 2010.
• "Google BigQuery - Using BigQuery from a Google Spreadsheet". 2010.

External links

• Official website
• Intro to new Google cloud technologies: Google Storage, Prediction API, BigQuery slideshare presentation by Chris Schalk (Developer Advocate at Google)
• BigQuery, meet Google Spreadsheets
• R client library for communicating with Google BigQuery
• Reddit's r/bigquery for news and discussion.
• bigqueri.es for sharing queries and datasets.