RapCache - A ServiceNow Key-Value Pair Cache

If you have been around ServiceNow for a while, you know with a large degree of confidence that ServiceNow is a Relational Database with a sexy GUI. Toss in a workflow engine, some eventing, sprinkle in a few other goodies, and “presto!” you can make work do a little flow for you here and there. 

ServiceNow is really good at taking instruction from a single place and executing it in another single place while enabling people to poke at the work as it goes from point A to point B. That instruction may come from a Script Include or it may come from a server in Grandma’s secret lair (trust me, she has one). What ServiceNow did not intend to be is a high-performing developer-friendly key-value store. But, if it could be, why would we want it to be?

As a Developer or Engineer, your solution is always limited to the tools you have available and in ServiceNow, there are already many ways to get the job done. The same holds true when I start a home improvement project and I consider the tools I have in my garage. Even though I already have them, it doesn’t stop me from walking around Home Depot thinking of all the new projects I could start with all the tools on display. In that same way I wonder what I can build while perusing AWS services. Safe to say, Home Depot purchases were made and now I am a better handyman for it and my wife loves me more for it. See the pattern? Robert California agrees.‍

So what on Earth did I do?‍

I created the RapCache App. A very simple app that you install in your ServiceNow platform to play with. It will sit there for a bit until a problem appears in your workshop that needs resolving and it will hit you. “Holy smokes! I installed an app a couple of weeks ago that will efficiently enable me to effectively do that thing you want, Boss!”. And they will love you more for it. See the pattern?‍

“But why do we need a new tool?”‍

A standardized key-value store and caching mechanism is great for several reasons:

1. Your app needs to quickly access unstructured data many times. Pop the data into memory and commit it to disk only when you need to.
2. You are orchestrating tasks from a main script across available nodes and you need to transfer data between the workers and/or back to the main script.
3. You have an asynchronous event driven architecture and you want to easily share data between various transactions running at very different times.
4. Some components of your app are simple, they don’t require structured data and don’t require long term persistent storage.
5. You want to seamlessly communicate with processes running in memory on the server side, from processes running in memory on the client side.
6. And many more reasons…

So here is to the new hammer in your tool belt; let’s rock and roll!‍

Install:

Go install the app on your instance then come back for some fun (You’ll see the update set in the repo):

https://github.com/rapdev-io/servicenow-oss/tree/main/rapcache

Overview:

The RapCache App consists of the following important objects:  

1) Cache Table and a Key Value Pair Column

2) Script Include

3) Scripted Rest Service with resources

4) A safety mechanism to delete old caches because... Developers

Create a Cache  

In ServiceNow:

‍

 In Python‍

Setting Key-Value Pairs

Now we can go retrieve that value with this

It’s convenient to throw many kv pairs into a single referrable cache

Cleaning things up

Performance

It takes less than 20 seconds to fumble around with 1000 of these.  

Since, by design we are not searching by key, and ever only searching by sys_id of a cache then there is no table scanning of the key value pairs. Any searching/looping/handling of keys and their values would be done by your logic in memory.

If you can’t make up your mind about which show is best, it improves performance by 5x making modifications in memory and waiting to commit modifications to the table until the end. And of course with a larger table, performance gains will be much more (remember that time you tried to query an incident table with millions of records in it?).

So there you have it. A standardized, convenient and efficient way to operate with unstructured data in memory while still getting the benefits of a persistent data store within ServiceNow. 

RapCache by RapDev.

*disclaimer - I’ve provided background scripts, everyone is having fun until you take out a production server. And remember to use your own sys_ids ;)

What’s next?

I would really like to see a python module and a node package with some convenient methods to tap into your ServiceNow’s new key-value store. Who knows, I might just build them soon.

But I would also be interested to see a simple-to-use client side script enabling devs to tap into the server side cache. With UI scripts being a ‘scoped app whoopsie daisy’, I’ll need a couple more beers to think this one through to conserve your precious keystrokes.