Running Query from text file

Asked at 2017-01-10 13:33:51Z
  • 5 Subscribers

I'm trying to run a big query query from the command line, but because my query is very long I've written it in a text file. The query works from the GUI and I'm overwriting a table that already exsists

bq query --allow_large_results --replace --destination_table=me.Tbl_MyTable  '`cat query.txt`'

However, I'm getting error results:

Error in query string: Error processing job 'dev:bqjob_r_00000123456789456123_1': Encountered " "\'cat query.txt\' "" at line 1, column 1. Was expecting: EOF

  • Do I need to put the entire file path in the .txt filename? (this doesn't seem to make a difference)
  • Are there any characters I need to be careful with in the text file (e.g. "\" or quotation marks) ?
  • I'm using where clauses and group by clauses - is that an issue?

2 answers in total

Elliott Brossard Posted at 2017-01-10 14:30:58Z

Instead of cat, just pipe the input from the file. The command would be:

bq query --allow_large_results --replace --destination_table=me.Tbl_MyTable < query.txt

This will send the contents of query.txt to the bq tool.

Felipe Hoffa Posted at 2017-01-11 03:27:22Z

Elliot is right, now if you want to cat, sed or anything, pipe it:

cat query.txt | bq query

Answer this questsion