External Storage - Google Cloud Storage

External Storage / Google Cloud Storage

On this page:

Google Cloud Storage API offers two different methods of authentication - HMAC Developer Keys and JSON Keyfiles. Optidash supports both methods, however if you'd like to authenticate with JSON Keyfiles you would first need to upload them to your Optidash Account (read below).

Authentication with HMAC Developer Keys

When using Developer Keys to authenticate with Google Cloud Storage (which is AWS S3-compatible) you only need to provide your developer key and secret (Storage Access Keys) within the store hash. The rest of store parameters is the same as with JSON Keyfiles authentication.

In order to access and generate Google Developer Keys you must have "Interoperability" enabled.

To do so, open the Cloud Storage Settings page in the Google Cloud Platform Console and select Interoperability tab. Than, if you have not set up interoperability before, click "Enable interoperability access" and than "Create a new key".

Authentication with JSON Keyfiles

This method of authentication with Google Cloud Storage relies on JSON Keyfiles which are bigger chunks of JSON-formatted project properties such as "type", "project_id" and "private_key_id". Those Keyfiles are on average 2.5 KB each and transmitting that much data with every request to the Optidash API would be a waste of bandwidth and could complicate your application.

In order to use JSON API from Google Cloud Storage, you must first upload your Google Keyfile to your Optidash Account; we encrypt the file while it is on our servers. When instructing the Optidash API to use Google Cloud Storage, you only have to specify a unique Keyfile ID that we internally assign to your uploaded Keyfile.

To generate and retrieve JSON Keyfiles for Google Cloud Storage the following Google APIs must to be enabled: Google Cloud Storage and Google Cloud Storage JSON API

Using Google Cloud Storage

In order to use Google Cloud Storage as your External Storage of choice you must set the provider attribute to google within the store hash and, at a minimum, provide either your unique ID of the JSON keyfile you uploaded to your Optidash Account (when using Keyfiles authentication) or key and secret (when using Developer Keys) as well as a bucket name. The full list of store hash attibutes for Google Cloud Storage can be found below.

store hash attributes for Google Cloud Storage:
AttributeTypeRequiredDescription
providerStringYesprovider must be set to google.
keyfileStringYesThe unique ID of the JSON Keyfile stored in your Optidash Account
(only when using Google Keyfiles authentication).
keyStringYesDeveloper Access Key
(only when using Developer Key authentication).
secretStringYesDeveloper Secret
(only when using Developer Key authentication).
bucketStringYesName of a destination bucket in your Google Cloud Storage account.
pathStringNoDestination path in your Google Cloud Storage bucket (without leading slash).
Defaults to root.
aclStringNoThe Access Control List in a JSON API-format of the destination object.
Defaults to "publicRead".
metadataHashNoCustom Metadata you would like to set on your object.
headersHashNoCustom HTTP headers you would like to set on your object.

The Optidash API allows you to set the following custom headers on your objects: Cache-Control, Content-Type, Content-Encoding, Content-Language and Content-Disposition.

When authenticating with JSON Keyfiles, store hash would look like so:

{
    "store": {
        "provider": "google",
        "keyfile": "id-of-json-keyfile",
        ..
    }
}

When authenticating with Developer Keys, store hash would look like so:

{
    "store": {
        "provider": "google",
        "key": "your-developer-key",
        "secret": "your-developer-secret",
        ..
    }
}

An example cURL request of using Google Cloud Storage (with JSON Keyfiles) as an External Storage provider will look like the following:

curl https://api.optidash.ai/1.0/fetch -X POST -u your-api-key: \
-H "Content-Type: application/json" \
-d '{
    "url": "https://www.website.com/image.jpg",
    "resize": {
        "width": 100,
        "height": 75
    },
    "store": {
        "provider": "google",
        "keyfile": "id-of-json-keyfile",
        "bucket": "images",
        "path": "assets/image.jpg",
        "acl": "publicRead",
        "metadata": {
            "key": "value"
        },
        "headers": {
            "Cache-Control": "max-age=2592000000"
        }
    }
}'

When using Google Cloud Storage as your External Storage, the url property within the JSON response will point to the object’s location within the Google Cloud Storage bucket and you can safely use that URL in production, for example:

HTTP/1.1 200 OK

Date: 
Status: 200 OK
Content-Type: application/json

{
    "success": true,
    "code": 200,
    "id": "5f17de07-d301-4154-9293-ba6d85285d39",
    "input": {
        "name": "image.jpg",
        ..
    },
    "output": {
        "url": "https://storage.googleapis.com/images/assets/image.jpg",
        ..
    }
}

Custom Bucket Permissions

If you would like to create a Google Cloud User dedicated only for the Optidash API or are using custom bucket permissions, please make sure that the user performing the API call has WRITER permissions.

Code Examples for Google Cloud Storage

Below, you'll find examples of how to use Google Cloud Storage (with JSON Keyfiles) from the Optidash API:

  • PHP
  • Node
  • Ruby
  • Go
  • Python
  • Java
<?php

/**
* Instantiate new `$opti` by calling a constructor
*/

$opti = new Optidash\Optidash('your-api-key');


/**
* Provide a publicly available image URL with fetch(string) method,
* resize the image to 100 x 75 and store it in your Google Cloud Storage
*/

$opti
    ->fetch('https://www.website.com/image.jpg')
    ->resize(array(
        'width' => 100,
        'height' => 75
    ))
    ->store(array(
        'provider' => 'google',
        'keyfile' => 'id-of-json-keyfile',
        'bucket' => 'images',
        'path' => 'assets/image.jpg',
        'acl' => 'publicRead',
        'metadata' => array(
            'key' => 'value'
        ),
        'headers' => array(
            'Cache-Control' => 'max-age=2592000000'
        )
    ))
    ->toJSON(function ($error, $meta) {
        if (!empty($error)) {
            throw new Exception($error);
        }

        /**
        * You'll find the full JSON metadata array within the `$meta` variable.
        * Remember to always check if the `success` property is set to `true`.
        */

        if ($meta['success'] == true) {
            print ($meta['output']['url']);
        } else {
            print ($meta['message']);
        }
    });
/**
* Module dependencies
*/

const Optidash = require("optidash");


/**
* Instantiate new `opti` by calling a constructor
*/

const opti = new Optidash("your-api-key");


/**
* Provide a publicly available image URL with fetch(string) method,
* resize the image to 100 x 75 and store it in your Google Cloud Storage
*/

opti
    .fetch("https://www.website.com/image.jpg")
    .resize({
        width: 100,
        height: 75
    })
    .store({
        provider: "google",
        keyfile: "id-of-json-keyfile",
        bucket: "images",
        path: "assets/image.jpg",
        acl: "publicRead",
        metadata: {
            key: "value"
        },
        headers: {
            "Cache-Control": "max-age=2592000000"
        }
    })
    .toJSON((err, meta) => {
        if (err) {
            return console.log(err);
        }

        /**
        * You'll find the full JSON metadata hash within the `meta` variable.
        * Remember to always check if the `success` property is set to `true`.
        */

        if (meta.success) {
            console.log(meta.output.url);
        } else {
            console.log(meta.message);
        }
    });
##
# Require dependencies

require "opti"


##
# Instantiate new `opti` by calling a constructor

opti = Optidash.new("your-api-key")


##
# Provide a publicly available image URL with fetch(string) method,
# resize the image to 100 x 75 and store it in your Google Cloud Storage

err, meta = opti
    .fetch("https://www.website.com/image.jpg")
    .resize(
        width: 100,
        height: 75
    )
    .store(
        provider: "google",
        keyfile: "id-of-json-keyfile",
        bucket: "images",
        path: "assets/image.jpg",
        acl: "publicRead",
        metadata: {
            key: "value"
        },
        headers: {
            "Cache-Control": "max-age=2592000000"
        }
    )
    .to_json

abort(err) if err


##
# You'll find the full JSON metadata hash within the `meta` variable.
# Remember to always check if the `success` property is set to `true`.

if meta["success"]
    puts meta["output"]["url"]
else
    puts meta["message"]
end
package main

import (
    "fmt"
    "github.com/optidash/optidash-go"
)

func main() {

    /**
    * Instantiate new `opti` by calling a constructor
    */

    opti, err := optidash.NewClient("your-api-key")

    if err != nil {
        panic(err)
    }


    /**
    * Provide a publicly available image URL with Fetch(string) method,
    * resize the image to 100 x 75 and store it in your Google Cloud Storage
    */

    meta, err := opti.
        Fetch("https://www.website.com/image.jpg").
        Resize(optidash.P{
            "width": 100,
            "height": 75
        }).
        Store(optidash.P{
            "provider": "google",
            "keyfile": "id-of-json-keyfile",
            "bucket": "images",
            "path": "assets/image.jpg",
            "acl": "publicRead",
            "metadata": optidash.P{
                "key": "value"
            },
            "headers": optidash.P{
                "Cache-Control": "max-age=2592000000"
            }
        }).
        ToJSON()

    if err != nil {
        panic(err)
    }


    /**
    * You'll find the full JSON metadata hash within the `meta` variable.
    * Remember to always check if the `success` property is set to `true`.
    */

    if meta["success"].(map[string]interface{})["success"].(bool) {
        fmt.Println(meta["output"].(map[string]interface{})["url"])
    } else {
        fmt.Println(meta["message"])
    }
}
##
# Import dependencies

from optidash import optidash


##
# Instantiate new `opti` by calling a constructor

opti = optidash('your-api-key')


##
# Provide a publicly available image URL with fetch(string) method,
# resize the image to 100 x 75 and store it in your Google Cloud Storage

err, meta = (
    opti
        .fetch('https://www.website.com/image.jpg')
        .resize({
            'width': 100,
            'height': 75
        })
        .store({
            'provider': 'google',
            'keyfile': 'id-of-json-keyfile',
            'bucket': 'images',
            'path': 'assets/image.jpg',
            'acl': 'publicRead',
            'metadata': {
                'key': 'value'
            },
            'headers': {
                'Cache-Control': 'max-age=2592000000'
            }
        })
        .toJSON()
)

if err is not None:
    raise StandardError(err)


##
# You'll find the full JSON metadata hash within the `meta` variable.
# Remember to always check if the `success` property is set to `true`.

if meta['success'] is True:
    print (meta['output']['url'])
else:
    print (meta['message'])
package com.optidash.examples;

/**
* Import dependencies
*/

import static com.optidash.OperationConfiguration.settings;
import com.optidash.Optidash;
import com.optidash.Fetch;
import com.optidash.Response;

public class StorageGoogle {
    public static void main(String[] args) {

        /**
        * Instantiate new `opti` by calling a constructor
        */

        Optidash opti = new Optidash("your-api-key");

        /**
        * Provide a publicly available image URL with fetch(string) method,
        * resize the image to 100 x 75 and store it in your Google Cloud Storage
        */

        final Fetch fetch = opti.fetch("https://www.website.com/image.jpg");

        fetch.resize(
            settings()
                .set("width", 100)
                .set("height", 75)
        );

        fetch.store(
            settings()
                .set("provider", "google")
                .set("keyfile", "id-of-json-keyfile")
                .set("bucket", "images")
                .set("path", "assets/image.jpg")
                .set("acl", "publicRead")
                .set("metadata", settings("key", "value"))
                .set("headers", settings()
                    .set("Cache-Control", "max-age=2592000000")
                )
        );

        final Response response = fetch.toJson();

        /**
        * You'll find the full JSON metadata hash within the `Response#getMetadata()` variable.
        * Remember to always check if the `Response#successful` property is set to `true`.
        */

        if (response.isSuccessful()) {
            System.out.println(response.getMetadata().getOutput().get("url"));
        } else {
            System.out.println(response.getMessage());
        }
    }
}
Grace Przemek Magda Damian
Have questions? Let's talk!We usually respond in just under an hour

Thanks for reaching out!

We've received your message
and will get to it as quickly as possible.

Whoops, looks like something went wrong.

We encountered an unexpected error and cannot complete your request. Please try reloading the page.