GCP Bigquery
- Project in GCP
- IAM Role which has access to big query service
- Navigate to
create forwarder
page - Select
bigquery
forwarder - Provide the credentials for the IAM role as JSON in
json_key
field - Provide
project_id
,dataset_id
andtable_id
- Set
table_expiration
in hours which will delete the table after the specified expiration time - Enable the
create_table
toggle to define the schema for the table - Add required columns for the table
- Provide
name
for the forwarder and clickcreate

- Select
bigquery
forwarder - Provide the credentials for the IAM role as JSON in
json_key
field - Provide
project_id
, existingdataset_id
and existingtable_id
- Set
table_expiration
in hours which will delete the table after the specified expiration time - Disable the
create_table
toggle to define the schema for the table - Provide
name
for the forwarder and clickcreate
- Navigate to
forwarders
page - Click
edit
icon in the big query forwarder for which you need to add columns in the schema - Click on
add column
to update the schema

When you forward your logs to BigQuery, the fields in the log will be mapped to the column in the table. By default, Apica maps the schema from the forwarder to the facet in the logs. It will insert the value for the facet as a value to the column in the table.

facets

Columns in bigquery table
To change the default mapping behavior, create a forward rule for the namespace and application by renaming the facet with the column name. This will help you to manage the mappings between your log and the BigQuery table schema.