Decoding Google Datastore’s URL Safe Key

Google Cloud Datastore is a highly scalable, managed NoSQL database service. I have used it for years to store millions of records. One of the things that I like about it; is the "Key". Because it has all kinds of meta information, not just the ID.

a Key (like a primary key) A unique identifier for an entity; it's usually a long number. But it's more than that. It's a complex object which contains the unique ID (64-bit integer), Entity Name, Path, Parent, etc Datastore also gives you. URL Safe Key; that you can safely pass around. You can easily find it in the console.

Google Datastore's URL Safe Key

It's not that difficult to decode it once you know how it's constructed. It's a base64 encoded protobuf of Reference. Sometimes the bas64 encoded string is compressed (the padding is removed). So you might have to calculate and add it. Usually one = or two == at the end.

The protobuf for Reference is below. Save it as key.proto

// Copyright 2017 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

syntax = "proto2";

message Reference {
  required string app = 13;
  optional string name_space = 20;
  required Path path = 14;
  optional string database_id = 23;
}

message Path {
  repeated group Element = 1 {
    required string type = 2;
    optional int64 id = 3;
    optional string name = 4;
  }
}

Then to decode you can just run

echo ag5qfnRoZWplc2hnbi1nY3IVCxIIY29udGFjdHMYgICA6NeHgQoM | \
base64 -d -i | \
protoc --decode=Reference key.proto

It throws the output Which has all the details.

app: "j~thejeshgn-gc"
path {
  Element {
    type: "contacts"
    id: 5634161670881280
  }
}

It's not difficult to generate one too. Just reverse the process.

You can read this blog using RSS Feed. But if you are the person who loves getting emails, then you can join my readers by signing up.

Join 2,132 other subscribers

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.