Content
- What is Google Sheet
- When Google Sheets is Not Enough for Data Analysis
- What is Google BigQuery Data Warehouse
- 3 Ways to Move Data from Google Sheets to BigQuery
- Option #1: Upload Sheets Data to BigQuery with OWOX BI Reports Extension
- Option #2: Transfer Data from Google Sheets using the BigQuery Interface
- Option #3: Connect to BigQuery from Google Sheets With Connected Sheets
How To Connect Google Sheets to BigQuery: 3 Ways
Ievgen Krasovytskyi, Head of Marketing @ OWOX
If you are looking for a convenient way to transfer data from Google Sheets into BigQuery, this article is for you. Learn how you can build BigQuery Tables with Google Sheets data.
We also have another article covering 3 ways to connect BigQuery to Google Sheets for building awesome reports so you can build visual reports on top of your corporate data, pivots, and charts in Sheets tables based on data from GBQ and how to avoid common 50,000-100,000 rows limitations, file sizes, or using CSV files.
Note: This post was originally published in November 2019 and was completely updated in March 2024 for accuracy and comprehensiveness on the state of BigQuery, Google Sheets, and connector services in 2024.
What is Google Sheet
It's difficult to find a marketer doing data analysis, a data professional, or any business spreadsheet user who doesn't work with Google products. And, of course, one of the most common is G Sheets.
It's a free tool, with many functions and built-in formulas, smart access control management, and it is very convenient to work with. You can create pivot tables, and charts, use Google Docs as well, and seamlessly connect to Google Cloud storage.
In addition, it's probably the best spreadsheet tool out there to collaborate with team members as well as other stakeholders whenever and wherever.
When Google Sheets is Not Enough for Data Analysis
Google Sheets Spreadsheet is a very simple and convenient tool for anybody within a small company, as well as for a huge enterprise to perform analysis. However, the use cases might be different.
If you don't have a lot of information for analysis, and the data you use is required by only a few team members, then it's difficult to find a better tool to build, basically, any reports, visualize, and share them.
However, as the company grows and data volumes increase (including the use of data from different sources), spreadsheets are still the best tools for ad-hoc reporting, to analyze data, but it's not suitable for storing the data (as sheet and cell range is getting to the point where PIVOT table populates slowly) or handling data preparations for reporting (like JOINs or VLOOKUPs) and that's exactly where modern data warehouse would help you streamline the data processes.
At this point, you might need to implement a data warehouse solution, just like BigQuery.
Dive deeper with this read
Google BigQuery: The Best Marketing Data Warehouse
What is Google BigQuery Data Warehouse
Google BigQuery cloud storage allows you to collect data from different sources, process it in seconds using SQL queries, and build reports with any metrics you need avoiding any restrictions such as data sampling in GA4 or simply slow report processing time in Sheets.
It's one of the most popular relationship management database systems out there and definitely one of the most suitable for marketing and digital analytics. Why? Because of the native integrations with other Google products such as GA4, Google Ads, Search Console, and at the end of the day Google Chrome and Workspace Marketplaces with thousands of apps there.
Simply put, you don't need to waste time finding external pipelines if you don't want to. You have a lot of out-of-the-box Google BigQuery connectors made by Google.
Here are a few of the other benefits of storing and processing data in BigQuery:
- It's very fast, and reliable and processes gigabytes of data in seconds;
- Continuous improvement and new features added by developers (over 155 new things announced for 2024);
- Out-of-the-box SQL queries;
- Strong machine learning (ML) and artificial intelligence (AI) capabilities;
- Fully serverless and doesn't force you to own your server.
- Sage, secure with control access management.
- BigQuery offers free credits to get started.
Uncover in-depth insights
Ultimate Data Quality Checklist
Download nowBonus for readers
3 Ways to Move Data from Google Sheets to BigQuery
If you want to use all the advanced analytics capabilities for marketing, finance and any other industry, sooner or later, you'll have a case when you need to transfer the data from Google Sheets to a data warehouse like BigQuery.
To connect data stored in Google Sheets to BigQuery, you can use one of the following sheets data connector methods:
Option #1: Upload Sheets Data to BigQuery with OWOX BI Reports Extension
Step 1: Install the extension from the Workspace Marketplace. You can type in BigQuery and just select the first one:
Step 2: Open a Google Sheet Spreadsheet you want to upload to BigQuery. (or upload a csv to Google Drive)
Step 3: Go to ‘Extensions' and select ‘OWOX BI BigQuery Reports' — ‘Upload data to BigQuery':
Note: If you are working with this extension for the first time, you'll need to select your GCP (Google Cloud Platform) Project ID.
Step 4: A new pop-up window opens. In the destination section - you need to select the project ID, and the dataset ID, and come up with a name for the table in which you want to load your single spreadsheet.
Note: If you need to create a new dataset first - you can do that in BigQuery UI.(You can find a detailed instruction about this below)
Step 5: Then, check the boxes for the columns you want to import.
Step 6: Next, we need to specify the data type for the fields. By default, every column has a ‘STRING', so you have to replace the types of data according to your business context.For example, for numeric identifiers, the type is ‘INTEGER' data type, for prices use the 'NUMERIC' data type or 'FLOAT', for dates - the 'DATE' data type, etc.
Step 7: Click Start Upload, and your document will be uploaded to BigQuery.
Note, when you reload data to the same table, you will see the message 'Table exists, choose write action': 'APPEND' or 'TRUNCATE'
Step 8: Great. Your Data is now Uploaded from Google Sheets to BigQuery table and you can click the link to check the ingested data in BigQuery UI right away.
Why use OWOX BI as Google Sheets to BigQuery Connector
Here are the top 4 reasons to choose OWOX BI BigQuery Reports Extension as a two-way Google Sheets to BigQuery connector:
- This Upload is free.
- Table schema auto-detection.
- The extension works both ways, meaning you can send data from BigQuery to Sheets as well.
- It's safe and secure. Only Google's official service APIs. All data is transferred to your Cloud Platform Project which you control access to, meaning your data never stays in the wrong hands.
Here are the 4 reasons to use OWOX BI as the BigQuery to Google Sheets Connector:
- You can quickly create reports and charts with any amount of data directly from Sheets (avoiding any of the row limitations).
- You can configure reports to be updated automatically at the frequency you want.
- A convenient query editor allows you to save and share queries with your team members so they can replicate your report.
- You can pre-specify dynamic parameters in a query that even people who are unfamiliar with SQL can apply to filter the query result.
- You can share the reports with stakeholders — just share the document. But also, the employees who have access to your project in GBQ will be able to re-use the queries you built. Plus, you can control the edit history for SQL queries.
- The extension works both ways, meaning that you can also send data from Sheets to GBQ.
Turn Data into Actionable Insights
Auto-generate reports and dashboards from your data in Google Sheets
...plus, it's 100% Free!
Option #2: Transfer Data from Google Sheets using the BigQuery Interface
Step 1: Navigate to your BigQuery account: Open your BigQuery console.
Step 2: Open the drop-down menu at the top of the BigQuery interface where the project is specified. [Skip this if you have the project prepared]
Step 3: Click on the 'New Project' button and create a new project. [Skip this if you have the project prepared]
Step 4: Give your project a name, select Billing Account, desired location, and click 'Create'. [Skip this if you have the project prepared]
Step 5: Select 'SELECT PROJECT' on the screen.
Step 6: Click on 3 dots near your project ID and ‘Create Dataset' on the right-hand side of your screen.
Step 7: Name your dataset (e.g., ‘SheetsDemo’), select Location Type, and click 'Create dataset'.
Step 8: Find your dataset, click on it, and tap on ‘CREATE TABLE’.
Step 9: Enter a new table name (e.g., ‘SheetsDemoTable’, and where it says ‘Create new table from’ open the drop-down.
Step 10: Select ‘Drive’ option
Step 11: Grab the URL from your Google Drive Doc.
Step 12: Enter the Drive URI, and click on the drop-down for File Format.
Step 13: Select ‘Google Sheet’ as a File format.
Step 14: Specify the list and the cell range in Google Sheets to determine where the data comes from for the new table in BigQuery. Ignore the heading row when doing this.
Note: If you entered an incorrect cell range then you may get an unexpecteddata when you query your table or your query may fail to execute.
Step 15: Now it’s time to define a schema for your table based on your Google Sheets data types.
Step 16: Click on the "Add field" button to start defining the schema for your table.
Step 17: Input the name of your field, choose the data type, and add the field to the schema.
Step 18: Repeat the process of adding fields to complete the schema.
Step 19: Create the table by clicking the 'Create table' button.
Your table is created, now let’s check whether your data is successfully imported.
Step 20: Select the table and use the 'QUERY’ button to start querying.
Step 21: Write a SQL statement: SELECT * FROM {project_id.dataset_id.table_name} and click 'Run'
Step 22: Handle any errors; if necessary, adjust your table or schema and recreate the table.
Optional step 23: Save your query results by selecting 'SAVE RESULTS' and choosing the save option.
Step 24: Store the results in BigQuery by selecting a table and clicking 'Save'.
Step 25: Preview your saved data by clicking on the 'Preview' tab.
Option #3: Connect to BigQuery from Google Sheets With Connected Sheets
Google has updated its Sheets to BigQuery Connector from Sheets. It's now called Connected Sheets and is designed for anybody to upload Google Sheets to BigQuery tables
To upload the required data to BigQuery, you'll need to:
Step 1: Open the spreadsheet you want to upload into BigQuery.
Step 2: Click the Data menu, choose the Data connectors line, and click on Connect to BigQuery.
Step 3: Click Get Connected.
Step 4: Select a Google Cloud project ID with billing enabled and click on it.
(If you don't find any projects, you need to add one)
Step 5: Choose a dataset (including the public datasets).
Step 6: Choose a table or view and click Connect. (You can pick from any company table you have access to or from a public dataset).
Step 7: Now your data is connected. You now can create connected charts, pivot tables, and functions that work on the entire dataset or extract a subset of the raw data directly into Google Sheets.
Dive deeper with this read
OWOX BI BigQuery Reports vs. Connected Sheets: Which is the Best Fit for Your Reporting Needs
Challenges using the Connect to BigQuery option
While Connected Sheets as a native Google Sheets connector offers a lot of advantages, there are notable challenges that users might encounter when using it:
- Size Limitation: There is a limit on the size of the data that can be uploaded at one time. If your document contains a large amount of information, you might need to break it into smaller chunks.
- Formatting Issues: There is no auto-detect schema here. Some fields in Google Sheets may not correspond directly to BigQuery data types, and you don't have any options to influence this while uploading the data. This leads to errors, loss, or poor data quality during the transfer.
- Error Handling: There is no error reporting or handling. That's why it's difficult to troubleshoot and resolve issues during the upload process.
- Another challenge is the need for a specific type of Workspace account. To use Connected Sheets Connector for BigQuery, users must have a Business, Enterprise, or Education GSuite account (an Education G Suite account would not work). These account types come with higher subscription costs than basic or standard Google Workspace accounts.
These challenges highlight the need for users to carefully consider their needs of analyzing data, as well as BigQuery account capabilities before relying on the Sheets Connector as the primary method for integrating Google Sheets to BigQuery.
Turn Data into Actionable Insights
Auto-generate reports and dashboards from your data in Google Sheets
...plus, it's 100% Free!
FAQ
-
How do I transfer data from Google Sheets to BigQuery?
Data transfer from Google Sheets to BigQuery can be accomplished through the Google Sheets to Bigquery Connector. This involves selecting a data set in BigQuery, using the "Create Table" option, selecting Drive as the source, and specifying your sheet's URL. Choose the appropriate file format and schema settings to complete the transfer.
-
How do I transfer data from BigQuery to Google Sheets?
Data transfer from Google Sheets to BigQuery can be accomplished through the Google Sheets to Google Bigquery Connector. This involves selecting a data set in BigQuery, using the "Create Table" option, selecting Google Drive as the source, and specifying your sheet's URL. Choose the appropriate file format and schema settings to complete the transfer. -
How do I create a table in BigQuery from Google Sheets?
To create a table in BigQuery from a Google Sheet, navigate to the BigQuery interface, select "Create Table" button, choose "Drive" as the source, and input the Google Sheet's URL. Select the file format as Google Sheets, configure the schema either manually or by auto-detection, and finalize by naming and creating the table. -
How do I create a Google BigQuery dataset?
Creating a dataset in Google BigQuery involves logging into the BigQuery console, selecting your project, and clicking "Create Dataset." You'll need to provide a unique dataset ID, choose a data location, and set data expiration terms if necessary. Once configured, click "Create Dataset" to finalize the setup. -
How to connect Google Sheets to BigQuery?
To connect Spreadsheets to BigQuery, use the BigQuery Data Connector in Google Sheets. Open your sheet, click on "Data" > "Data Connectors" > "Connect to BigQuery," and follow the prompts to select your BigQuery project and dataset. Or use OWOX BI BigQuery Reports Extension to simplify the process. This also allows you to query BigQuery data directly from within Google Sheets.
-
How to connect BigQuery to Google Sheets?
Connecting BigQuery to Google Sheets can be achieved through the Google Sheets Data Connector. Start by opening a Google Sheet, navigate to "Data" > "Data Connectors" > "Connect to BigQuery." Follow the instructions to link your BigQuery project. This setup enables direct querying and analysis of BigQuery data within Google Sheets. -
Is it secure to import sensitive data using the BigQuery Sheets Connector?
Yes, the BigQuery Sheets Connector uses Google Cloud's secure authentication framework, ensuring that only authorized users can access sensitive data. Additionally, all imported data is encrypted both in transit and at rest for maximum security. -
How do I install the BigQuery Sheets Connector?
To install the BigQuery Sheets Connector, simply navigate to the Google Sheets Extensions menu, click 'Extensions', then 'Get Add-ons'. Search for BigQuery and click 'install' on OWOX Reports, Charts & Pivots. Once it's complete, you should see a new OWOX Reports in the Extensions menu in your Google Sheets Extension toolbar.
-
What is the BigQuery Sheets Connector?
The BigQuery Sheets Connector is a tool that allows users to import data from Google Sheets directly into BigQuery, enabling data analysts to easily access crucial information and perform advanced analysis.