Comment on page
Google Pub/Sub
Google Pub/Sub is the GCP managed service for real-time stream processing
Apica's LogFlow can be used to push data to a PubSub topic with just a few clicks. You can forward data as is, morph the data by stripping down attributes or pick and choose the fields that you want to be sent to the Google PubSub topic
You will need the following before you can configure forwarding to Google PubSub
- 1.Name of the Google PubSub topic
- 2.Project id ( full name not an integer ) for your GCP account project where you created the topic
- 3.Service account access JSON
The service account must have pubsub.topics.publish permissions
You can check your service account json permissions with gcloud cli
$ gcloud auth activate-service-account --project=my-project-123 \
--key-file=sevice_account_key_file.json
$ gcloud pubsub topics publish my-gcp-topic-1 --message="hello"
messageIds:
- '6470211179713996'
$ gcloud pubsub subscriptions pull my-gcp-topic-1-sub --auto-ack
┌───────┬──────────────────┬──────────────┬────────────┬──────────────────┐
│ DATA │ MESSAGE_ID │ ORDERING_KEY │ ATTRIBUTES │ DELIVERY_ATTEMPT │
├───────┼──────────────────┼──────────────┼────────────┼──────────────────┤
│ hello │ 6470211179713996 │ │ │ │
└───────┴──────────────────┴──────────────┴────────────┴──────────────────┘
We are now ready to create the forwarder
Create Google PubSub forwarder
Forwarder configuration
You can now test a data replay with a test stream to see if data went through
Replay data to Google PubSub
Replay status for Google PubSub forwarded data
Visualizing data forwarding to Google PubSub