Skip to content

Commit

Permalink
docs(bigquery): link types on package docs (#11036)
Browse files Browse the repository at this point in the history 
Towards internal b/372529756 and b/372535179
  • Loading branch information
@alvarowolfx
alvarowolfx authored Oct 29, 2024
1 parent 2c83297 commit c4af6fe
Show file tree
Hide file tree
Showing 2 changed files with 47 additions and 44 deletions.
10 changes: 5 additions & 5 deletions bigquery/bigquery.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,19 +65,19 @@ type Client struct {
enableQueryPreview bool
}

// DetectProjectID is a sentinel value that instructs NewClient to detect the
// project ID. It is given in place of the projectID argument. NewClient will
// DetectProjectID is a sentinel value that instructs [NewClient] to detect the
// project ID. It is given in place of the projectID argument. [NewClient] will
// use the project ID from the given credentials or the default credentials
// (https://developers.google.com/accounts/docs/application-default-credentials)
// if no credentials were provided. When providing credentials, not all
// options will allow NewClient to extract the project ID. Specifically a JWT
// options will allow [NewClient] to extract the project ID. Specifically a JWT
// does not have the project ID encoded.
const DetectProjectID = "*detect-project-id*"

// NewClient constructs a new Client which can perform BigQuery operations.
// NewClient constructs a new [Client] which can perform BigQuery operations.
// Operations performed via the client are billed to the specified GCP project.
//
// If the project ID is set to DetectProjectID, NewClient will attempt to detect
// If the project ID is set to [DetectProjectID], NewClient will attempt to detect
// the project ID from credentials.
//
// This client supports enabling query-related preview features via environmental
Expand Down
81 changes: 42 additions & 39 deletions bigquery/doc.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ connection pooling and similar aspects of this package.
# Creating a Client
To start working with this package, create a client:
To start working with this package, create a client with [NewClient]:
ctx := context.Background()
client, err := bigquery.NewClient(ctx, projectID)
Expand All @@ -33,7 +33,7 @@ To start working with this package, create a client:
# Querying
To query existing tables, create a Query and call its Read method, which starts the
To query existing tables, create a [Client.Query] and call its [Query.Read] method, which starts the
query and waits for it to complete:
q := client.Query(`
Expand All @@ -52,7 +52,7 @@ query and waits for it to complete:
}
Then iterate through the resulting rows. You can store a row using
anything that implements the ValueLoader interface, or with a slice or map of bigquery.Value.
anything that implements the [ValueLoader] interface, or with a slice or map of [Value].
A slice is simplest:
for {
Expand Down Expand Up @@ -86,7 +86,7 @@ You can also use a struct whose exported fields match the query:
}
You can also start the query running and get the results later.
Create the query as above, but call Run instead of Read. This returns a Job,
Create the query as above, but call [Query.Run] instead of [Query.Read]. This returns a [Job],
which represents an asynchronous operation.
job, err := q.Run(ctx)
Expand All @@ -100,17 +100,17 @@ the results at a later time, even in another process.
jobID := job.ID()
fmt.Printf("The job ID is %s\n", jobID)
To retrieve the job's results from the ID, first look up the Job:
To retrieve the job's results from the ID, first look up the [Job] with the [Client.JobFromID] method:
job, err = client.JobFromID(ctx, jobID)
if err != nil {
// TODO: Handle error.
}
Use the Job.Read method to obtain an iterator, and loop over the rows.
Calling Query.Read is preferred for queries with a relatively small result set,
Use the [Job.Read] method to obtain an iterator, and loop over the rows.
Calling [Query.Read] is preferred for queries with a relatively small result set,
as it will call BigQuery jobs.query API for a optimized query path. If the query
doesn't meet that criteria, the method will just combine Query.Run and Job.Read.
doesn't meet that criteria, the method will just combine [Query.Run] and [Job.Read].
it, err = job.Read(ctx)
if err != nil {
Expand All @@ -120,26 +120,26 @@ doesn't meet that criteria, the method will just combine Query.Run and Job.Read.
# Datasets and Tables
You can refer to datasets in the client's project with the Dataset method, and
in other projects with the DatasetInProject method:
You can refer to datasets in the client's project with the [Client.Dataset] method, and
in other projects with the [Client.DatasetInProject] method:
myDataset := client.Dataset("my_dataset")
yourDataset := client.DatasetInProject("your-project-id", "your_dataset")
These methods create references to datasets, not the datasets themselves. You can have
a dataset reference even if the dataset doesn't exist yet. Use Dataset.Create to
a dataset reference even if the dataset doesn't exist yet. Use [Dataset.Create] to
create a dataset from a reference:
if err := myDataset.Create(ctx, nil); err != nil {
// TODO: Handle error.
}
You can refer to tables with Dataset.Table. Like bigquery.Dataset, bigquery.Table is a reference
You can refer to tables with [Dataset.Table]. Like [Dataset], [Table] is a reference
to an object in BigQuery that may or may not exist.
table := myDataset.Table("my_table")
You can create, delete and update the metadata of tables with methods on Table.
You can create, delete and update the metadata of tables with methods on [Table].
For instance, you could create a temporary table with:
err = myDataset.Table("temp").Create(ctx, &bigquery.TableMetadata{
Expand All @@ -153,15 +153,15 @@ We'll see how to create a table with a schema in the next section.
# Schemas
There are two ways to construct schemas with this package.
You can build a schema by hand, like so:
You can build a schema by hand with the [Schema] struct, like so:
schema1 := bigquery.Schema{
{Name: "Name", Required: true, Type: bigquery.StringFieldType},
{Name: "Grades", Repeated: true, Type: bigquery.IntegerFieldType},
{Name: "Optional", Required: false, Type: bigquery.IntegerFieldType},
}
Or you can infer the schema from a struct:
Or you can infer the schema from a struct with the [InferSchema] method:
type student struct {
Name string
Expand All @@ -174,10 +174,10 @@ Or you can infer the schema from a struct:
}
// schema1 and schema2 are identical.
Struct inference supports tags like those of the encoding/json package, so you can
Struct inference supports tags like those of the [encoding/json] package, so you can
change names, ignore fields, or mark a field as nullable (non-required). Fields
declared as one of the Null types (NullInt64, NullFloat64, NullString, NullBool,
NullTimestamp, NullDate, NullTime, NullDateTime, and NullGeography) are
declared as one of the Null types ([NullInt64], [NullFloat64], [NullString], [NullBool],
[NullTimestamp], [NullDate], [NullTime], [NullDateTime], [NullGeography], and [NullJSON]) are
automatically inferred as nullable, so the "nullable" tag is only needed for []byte,
*big.Rat and pointer-to-struct fields.
Expand All @@ -193,16 +193,17 @@ automatically inferred as nullable, so the "nullable" tag is only needed for []b
}
// schema3 has required fields "full_name" and "Grade", and nullable BYTES field "Optional".
Having constructed a schema, you can create a table with it like so:
Having constructed a schema, you can create a table with it using the [Table.Create] method like so:
if err := table.Create(ctx, &bigquery.TableMetadata{Schema: schema1}); err != nil {
// TODO: Handle error.
}
# Copying
You can copy one or more tables to another table. Begin by constructing a Copier
describing the copy. Then set any desired copy options, and finally call Run to get a Job:
You can copy one or more tables to another table. Begin by constructing a [Copier]
describing the copy using the [Table.CopierFrom]. Then set any desired copy options,
and finally call [Copier.Run] to get a [Job]:
copier := myDataset.Table("dest").CopierFrom(myDataset.Table("src"))
copier.WriteDisposition = bigquery.WriteTruncate
Expand All @@ -211,21 +212,21 @@ describing the copy. Then set any desired copy options, and finally call Run to
// TODO: Handle error.
}
You can chain the call to Run if you don't want to set options:
You can chain the call to [Copier.Run] if you don't want to set options:
job, err = myDataset.Table("dest").CopierFrom(myDataset.Table("src")).Run(ctx)
if err != nil {
// TODO: Handle error.
}
You can wait for your job to complete:
You can wait for your job to complete with the [Job.Wait] method:
status, err := job.Wait(ctx)
if err != nil {
// TODO: Handle error.
}
Job.Wait polls with exponential backoff. You can also poll yourself, if you
[Job.Wait] polls with exponential backoff. You can also poll yourself, if you
wish:
for {
Expand All @@ -247,8 +248,9 @@ wish:
There are two ways to populate a table with this package: load the data from a Google Cloud Storage
object, or upload rows directly from your program.
For loading, first create a GCSReference, configuring it if desired. Then make a Loader, optionally configure
it as well, and call its Run method.
For loading, first create a [GCSReference] with the [NewGCSReference] method, configuring it if desired.
Then make a [Loader] from a table with the [Table.LoaderFrom] method with the reference,
optionally configure it as well, and call its [Loader.Run] method.
gcsRef := bigquery.NewGCSReference("gs://my-bucket/my-object")
gcsRef.AllowJaggedRows = true
Expand All @@ -257,8 +259,9 @@ it as well, and call its Run method.
job, err = loader.Run(ctx)
// Poll the job for completion if desired, as above.
To upload, first define a type that implements the ValueSaver interface, which has a single method named Save.
Then create an Inserter, and call its Put method with a slice of values.
To upload, first define a type that implements the [ValueSaver] interface, which has
a single method named Save. Then create an [Inserter], and call its [Inserter.Put]
method with a slice of values.
type Item struct {
Name string
Expand Down Expand Up @@ -286,7 +289,7 @@ Then create an Inserter, and call its Put method with a slice of values.
// TODO: Handle error.
}
You can also upload a struct that doesn't implement ValueSaver. Use the StructSaver type
You can also upload a struct that doesn't implement [ValueSaver]. Use the [StructSaver] type
to specify the schema and insert ID by hand:
type item struct {
Expand Down Expand Up @@ -324,13 +327,13 @@ Lastly, but not least, you can just supply the struct or struct pointer directly
}
BigQuery allows for higher throughput when omitting insertion IDs. To enable this,
specify the sentinel `NoDedupeID` value for the insertion ID when implementing a ValueSaver.
specify the sentinel [NoDedupeID] value for the insertion ID when implementing a [ValueSaver].
# Extracting
If you've been following so far, extracting data from a BigQuery table
into a Google Cloud Storage object will feel familiar. First create an
Extractor, then optionally configure it, and lastly call its Run method.
[Extractor], then optionally configure it, and lastly call its [Extractor.Run] method.
extractor := table.ExtractorTo(gcsRef)
extractor.DisableHeader = true
Expand All @@ -339,16 +342,16 @@ Extractor, then optionally configure it, and lastly call its Run method.
# Errors
Errors returned by this client are often of the type googleapi.Error: https://godoc.org/google.golang.org/api/googleapi#Error
Errors returned by this client are often of the type [googleapi.Error].
These errors can be introspected for more information by using [errors.As]
with the richer [googleapi.Error] type. For example:
These errors can be introspected for more information by using `xerrors.As` with the richer *googleapi.Error type. For example:
var e *googleapi.Error
if ok := xerrors.As(err, &e); ok {
if e.Code == 409 { ... }
}
var e *googleapi.Error
if ok := errors.As(err, &e); ok {
if e.Code == 409 { ... }
}
In some cases, your client may received unstructured googleapi.Error error responses. In such cases, it is likely that
In some cases, your client may received unstructured [googleapi.Error] error responses. In such cases, it is likely that
you have exceeded BigQuery request limits, documented at: https://cloud.google.com/bigquery/quotas
*/
package bigquery // import "cloud.google.com/go/bigquery"

0 comments on commit c4af6fe

Please sign in to comment.