• 5 Subscribers
  • 114 Views
0

I am using BIGQUERY in C#, and trying save the query by insert command to temporary table.

When I am running the command j.Insert the insert is done asynchronously (as following code), and I cannot guarantee that the data is actually inserted completely.

Is there any way checking the status of the current running job, for knowing whenever the job has been finished or not (Wait that job is finished).

Here is my code (based on example on: Create a table from Select Bigquery

I add sleep(5000) in order to guartantee that the job has been finished, but this is a deterministic solution.

                // _bigQueryService is of type: BigQueryService.
                // query is the main query (select ...)
                JobsResource j = _bigQueryService.Jobs;

            Job theJob = new Job();

            DateTime n = DateTime.UtcNow;

            TableReference _destTempTable = new TableReference
            {
                ProjectId = "myprojectid",
                DatasetId = "myDataSetId",
                TableId = "myTempTable")
            };
            theJob.Configuration = new JobConfiguration()
            {
                Query = new JobConfigurationQuery()
                {
                    AllowLargeResults = true,
                    CreateDisposition =  "CREATE_IF_NEEDED", /* CREATE_IF_NEEDED, CREATE_NEVER */
                    DefaultDataset = "myDefaultDataSet",
                    MaximumBillingTier = 100,
                    DestinationTable = _destTempTable,
                    Query = query,
                }
            };
            var resJob = j.Insert(theJob, _settings.ProjetId).Execute();
            Thread.Sleep(5000); // *** I need better solution instead this line ****

1 answers in total

1
Elliott Brossard Posted at 2017-01-10 16:35:14Z

You can use PollUntilCompleted() to check for completion rather than Thread.Sleep(). In your case, it would be:

var resJob = j.Insert(theJob, _settings.ProjetId).Execute();
resJob.PollUntilCompleted();

You can see the complete sample on Github, but the relevant part linked from Querying Data is:

public BigQueryResults AsyncQuery(string projectId, string datasetId, string tableId,
    string query, BigQueryClient client)
{
    var table = client.GetTable(projectId, datasetId, tableId);
    BigQueryJob job = client.CreateQueryJob(query,
        new CreateQueryJobOptions { UseQueryCache = false });

    // Wait for the job to complete.
    job.PollUntilCompleted();

    // Then we can fetch the results, either via the job or by accessing
    // the destination table.
    return client.GetQueryResults(job.Reference.JobId);
}

It looks like in the new Google Cloud library for C#, it might be called PollQueryUntilCompleted based on the documentation.

Answer this questsion