Skip to main content

Configure GCP to Sync Media to Your Account

Get set up to sync media from a location on Google Cloud Platform (GCP)

T
Written by Taylor Maggos
Updated over 8 months ago

If your media are stored in a location you can access on GCP Cloud Storage, you can link that cloud location to your platform account and sync media to use in the platform.

Please note: this is our "digest" version of how to manage access for your GCP account. For the most up-to-date information on how to set up and manage access controls for GCP, please visit the GCP documentation center.

Before you start

You will need:

  • A GCP account. Open console.cloud.google.com and either log in or create an account.

  • A GCP project.

  • One or more GCP Storage buckets containing data that you wish to upload to CrowdAI. You can browse these buckets via the top-level hamburger menu.

  • Adequate permissions/roles.

    • If you created the GCP account, you will likely have the “Owner” role, which provides more than enough permissions.

    • The “Editor” role also contains sufficient permissions.

Set up a Saab, Inc. CV Platform service account via the GCP console

1. Create a platform role

A “role” is a collection of permissions. Here, we define a role that provides the permissions necessary for use by the platform (and nothing further).

This step only needs to be completed once per GCP project.

  1. Navigate to the roles page: Click on the top-left hamburger -> IAM & Admin -> Roles

  2. Click the + Create Role button in the top bar.

  3. Fill out the top part of the Create Role form

    1. A name like saab-credentials will help you to remember the context for this role.

    2. “Alpha” is fine for now.

  4. Click the + Add Permissions button

  5. In the permissions modal, select the following read-only permissions:

    1. storage.objects.list

    2. storage.objects.get

    3. Storage.buckets.list (candidate for removal)

    4. storage.buckets.get (candidate for removal)

  6. Click the Create button.

You should be redirected to the list of all roles. Our newly created saab-credentials role can be seen at the top.

2. Create a service account

We now need to create a service account, which is a sort of “machine user" that can access your GCP storage.

  1. Navigate to the service accounts page: Click on the top-left hamburger -> IAM & Admin -> Service accounts

  2. Click the + Create a new Service Account button in the top bar of the page.

  3. Fill in the fields. Click Create and Continue when ready.

  4. We need to assign the saab-credentials role from the previous step. Additionally, we will likely want to add “constraints.”

    1. The saab-credentials role can be found under the “custom” sub-menu. Select it.

    2. Now click the Add condition link directly next to the role selection.

  5. [Optional] Add a condition on the service key. Right now, our service key will be able to access all objects in all buckets of the GCP project. Oftentimes, though, you only want to expose one or a few buckets. This is where conditions come into play.

    1. Add a title and description.

    2. For copy+paste purposes, it will be easiest to use the “Condition Editor” tab.

    3. To expose only select buckets, use the following pattern:

      1. resource.name.startsWith("projects/_/buckets/saab-test-2") or resource.name.startsWith("projects/_/buckets/saab-test")

      2. The above example can be read as, “only objects from buckets saab-test’ OR saab-test-2 may be accessed.”

      3. We could further constrain the selection by adding a path. For example, projects/_/buckets/saab-test-2/inner only exposes files within the “inner” directory of bucket saab-test-2.

  6. Step 3 (“Grant users access to this service account”) can be skipped for now.

  7. Click the blue Done button at the bottom of the form.

You should be redirected to the service accounts list page. The newly created service account should appear in the list.

3. Create service key

Service keys contain the actual access information that we need to pass to the platform. Each service account may have several service keys attached to it. Moreover, these service keys may expire. These features are helpful in case a key is compromised: simply delete the compromised key and create a new one.

  1. Find the service account in the list of service accounts. Click the vertical dots in the far-right Action column. Select Manage keys.

  2. We are now in the “Keys” tab, which contains a list of keys.

  3. To create a new key, select Add key -> Create new key

  4. A Create key modal will appear. Select JSON (the default). Click Create.

  5. You will be prompted to download a JSON file. This file contains the “keys to the kingdom” (or, at least, part of it). So be careful:

    • If you intend to upload the JSON file to CrowdAI immediately, it would actually be preferable to “open” rather than “save” it. This will ensure that the information is deleted whenever you close your browser.

    • If you save the JSON file (e.g. because your browser automatically saves files), you should delete the file immediately after adding it to your platform account.

    • You should use this key to generate or update one set of credentials and nothing further. If, for example, you want to use GCP developer tools, then you should create a different service key.

With the JSON file in hand, you're ready to use it to create a set of Credentials in your platform account. Here's our article on how to do that.

Did this answer your question?