Redis is an in-memory key-value store that can be used as a database, cache, and message broker. The project isopen sourceand it's currently licensed under theBSD license.
Fun Fact: Redis means "REmote DIctionary Server."
Redis delivers sub-millisecond response timesthat enable millions of requests per second to power demanding real-time applications such as games, ad brokers, financial dashboards, and many more!
It supports basic data structures such as strings, lists, sets, sorted sets with range queries, and hashes. More advanced data structures like bitmaps, hyperloglogs, and geospatial indexes with radius queries are also supported.
Alex Stanciu, a Product Owner from the Identity Governance Team at Auth0, explains one of our use cases for Redis:
"We use Redis as a caching layer and a session store for our Slack Bot conversation engine. Because it stores data in memory (RAM), it provides ultra-fast read and write speeds; responses are usually in the single-digit milliseconds."
In this Redis tutorial, we'll learn how to set up Redis in our systems and how to store data in Redis using its core and most frequently used data structures. With this foundation, in future posts we'll learn how to use Redis for caching, session storage, messaging, and real-time analytics. Let's get started!
Installing Redis
The first thing that we need to do is install Redis. If you have it already running in your system, feel free to skip this part of the post.
TheRedis documentationrecommends installing Redis by compiling it from sources as Redis has no dependencies other than a workingGCCcompilerandlibc. We can either download the latest Redis tarball fromredis.io, or we can use a special URL that always points to the latest stable Redis version:http://download.redis.io/redis-stable.tar.gz.
Windows users: The Redis project does not officially support Windows. But, if you are running Windows 10, you canInstall the Windows Subsystem for Linuxto install and run Redis. When you have the Windows Subsystem for Linux up and running, please follow any steps in this post that apply to Linux (when specified) from within your Linux shell.
To compile Redis follow these simple steps:
Create aredisdirectory and make it the current working directory:
Make the unpackedredis-stabledirectory the current working directory:
macOS/Linux:
cd redis-stable
Compile Redis:
macOS/Linux:
make
If themakepackage is not installed in your system, please follow the instructions provided by the CLI to install it. In macOS, you may need to download XCode to have access to the command line tools which includemakeand a C compiler. For a fresh installation of Ubuntu, for example, you may want to run the following commands to update the package manager and install core packages:
tcl8.5 or newer is needed to run the Redis test in the next step.
Test that the build works correctly:
macOS/Linux:
make test
Once the compilation is done, thesrcdirectory withinredis-stableis populated with different executables that are part of Redis. The Redis docs explain thefunctionality of each Redis exectuble:
redis-server: runs the Redis Server itself.
redis-sentinel: runsRedis Sentinel, a tool for monitoring and failover.
redis-check-aofandredis-check-dump: used for the rare cases when there are corrupted data files.
We are going to be using theredis-serverandredis-cliexecutable frequently. For convenience, let's copy both to a location that will let us access them system-wide. This can be done manually by running:
While havingredis-stableas the current working directory, this can also be done automatically by running the following command:
macOS/Linux:
sudo make install
We need to restart our shell for these changes to take effect. Once we do that, we are ready to start running Redis.
Running Redis
Starting Redis
The easiest way to start the Redis server is by running theredis-servercommand. In a fresh shell window, type:
redis-server
If everything is working as expected, the shell will receive as outline a giant ASCII Redis logo that shows the Redis version installed, the running mode, theportwhere the server is running and it'sPID(process identification number).
We started Redis without any explicit configuration file; therefore, we'll be using the internal default configuration. This is acceptable for the scope of this blog post: understanding and using the basic Redis data structures.
As a first step, we always need to get the Redis server running as the CLI and other services depend on it to work.
How to Check if Redis is Working
As noted in the Redis docs, external programs talk to Redis using aTCP socket and a Redis specific protocol. The Redis protocol is implemented by Redis client libraries written in many programming languages, like JavaScript. But we don't need to use a client library directly to interact with Redis. We can use theredis-clito send a command to it directly. To test that Redis is working properly, let's send it thepingcommand. Open a new shell window and execute the following command:
redis-cli ping
If everything is working well, we should getPONGas a reply in the shell.
When we issuedredis-cli ping, we invoked theredis-cliexecutable followed by a command name,ping. A command name and its arguments are sent to the Redis instance running onlocalhost:6379for it to be processed and send a reply.
The host and port of the instance can be changed. Use the--helpoption to check all the commands that can be used withredis-cli:
redis-cli --help
If we runredis-cliwithout any arguments, the program will start in interactive mode. Similar to theRead–Eval–Print Loop (REPL)of programming languages like Python, we can type different Redis commands in the shell and get a reply from the Redis instance. What those commands are and what they do is the core learning objective of this post!
Let start first by learning how to manipulate data in Redis using commands!
"Redis is a key-value store that let us store some data, the value, inside a key. It offers ultra-fast performance to satisfy demanding real-time applications like video games 🎮."
As we learned earlier, Redis is a key-value store that let us associate some data called avaluewith akey. We can later retrieve the stored data if we know theexactkey that was used to store it.
In case that you haven't done so already, run the Redis CLI in interactive mode by executing the following command:
redis-cli
We'll know the interactive CLI is working when we see the Redis instance host and port in the shell prompt:
127.0.0.1:6379>
Once there, we are ready to issue commands.
Writing Data
To store a value in Redis, we can use theSETcommandwhich has the following signature:
SET key value
In English, it reads like "set key to hold value." It's important to note that if the key already holds a value,SETwill overwrite it no matter what.
Let's look at an example. In the interactive shell type:
SET service "auth0"
Notice how, as you type, the interactive shell suggests the required and optional arguments for the Redis command.
Pressenterto send the command. Once Redis stores"auth0"as the value ofservice, it replies withOK, letting us know that everything went well. Thank you, Redis!
Reading Data
We can use theGETcommandto ask Redis for the value of a key:
GET key
Let's retrieve the value ofservice:
GET service
Redis replies with"auth0".
What if we ask for the value of a key that has never been set?
GET users
Redis replies with(nil)to let us know that the key doesn't exist in memory.
In a classic API that connects to a database, we'd like to performCRUDoperations: create, read, update, and delete. We have covered how to create (write) and read data in Redis by using theSETandGETcommands respectively. Let's cover the rest.
Updating Data
We can update the value of a key simply by overwriting its data as mentioned earlier.
Lets create a new key-value pair:
SET framework angular
But, we change our mind and now we want the value to be"react". We can overwrite it like this:
SET framework react
Did Redis get it right? Let's ask for it!
GET framework
Redis indeed replies with"react". We are being a bit indecisive and now we want to set theframeworkkey to hold the value of"vue":
SET framework vue
If we runGETframeworkagain, we get"vue". The update/overwrite works as excepted.
Deleting Data
But, we don't want to actually set any framework for now and we need to delete that key. How do we do it? We use theDELcommand:
DEL key
Let's run it:
DEL framework
Redis replies with(integer)1to let us know the number of keys that were removed.
With just three commands,SET,GET, andDEL, we are able to comply with the four CRUD operations!
Wrapping Strings with Quotation Marks
Notice something curious: we did not have to put quotation marks around a single string value we wanted to store.SETframework angularandSETframework"angular"are both accepted by Redis as an operation to store the string"angular"as the value of the keyframework.
Redis automatically wraps single string arguments in quotation marks. Since both key and value are strings, the same applies for the key name. We could have usedSET"framework"angularand it would have worked as well. However, if we plan to use more than one string as the key or value, we do need to wrap the strings in quotation marks:
SET "the frameworks" "angular vue react"
Replies withOK.
SET the frameworks "angular vue react"
Replies with(error)ERRsyntax error
SET "the frameworks" angular vue react
Also replies with(error)ERRsyntax error
Finally, to retrieve the value, we must use the exact key string:
GET "the frameworks"
Replies with"angular vue react".
Non-Destructive Write
Redis is compassionate and lets us write data with care. Imagine that we wanted to create aserviceskey to hold the value"heroku aws"but instead of typingSETservices"heroku aws", we typedSETservice"heroku aws". This last command would overwrite the current value ofservicewithout mercy. However, Redis gives us a non-destructive version ofSETcalledSETNX:
SETNX key value
SETNXcreates a key in memory if and only if the key does not exist already (SETifNot eXists). If the key already exists, Redis replies with0to indicate a failure to store the key-value pair and with1to indicate success. Let's try out this previous scenario but usingSETNXinstead ofSET:
SETNX service "heroku aws"
The reply is(integer)0as we expected.
SETNX services "heroku aws"
This time, the reply is(integer)1. Great!
We can useSETNXto prevent us from mutating data accidentally.
Expiring Keys
When creating a key with Redis, we can specify how long that key should stay stored in memory. Using theEXPIREcommand, we can set a timeout on a key and have the key be automatically deleted once the timeout expires:
EXPIRE key seconds
Let's create anotificationkey that we want to delete after 30 seconds:
SET notification "Anomaly detected"
EXPIRE notification 30
This schedules thenotificationkey to be deleted in 30 seconds. We could look at a clock and check after 30 seconds have elapsed ifnotificationis still available but we don't have to do that! Redis offers theTTLcommand that tells us how many seconds a key has left before it expires and gets deleted:
TTL key
It's possible that more than 30 seconds have already passed so let's try the above example again, but this time callingTTLas soon as we executeEXPIRE:
SET notification "Anomaly detected"
EXPIRE notification 30
TTL notification
Redis replied with(integer)27to me, indicating thatnotificationis still available for 27 more seconds. Let's wait a bit and runTTLagain:
TTL notification
This time, Redis replied with(integer)-2. Starting with Redis 2.8,TTLreturns:
The timeout left in seconds.
-2if the key doesn't exist (either it has not been created or it was deleted).
-1if the key exists but has no expiry set.
I made sure that 30 seconds had passed so-2was expected. Let's see the error message when the key exists but has no expiry set:
SET dialog "Continue?"
TTL dialog
As expected, with no expiry set, Redis replies with(integer)-1.
It's important to note that we can reset the timeout by usingSETwith the key again:
SET notification "Anomaly detected"
EXPIRE notification 30
TTL notification
// (integer) 27
SET notification "No anomaly detected"
TTL notification
// (integer) -1
We learned earlier that usingSETis the same as creating the key again, which for Redis also involves resetting any timeouts currently assigned to it.
We have a solid foundation now on manipulating data in Redis. With this knowledge under our belt, we are ready to now explore the data types that Redis offers.
Redis Data Types
Far from being a plain key-value store, Redis is an actualdata structure serverthat supports different kinds of values. Traditionally, key-value stores allow us to map a string key to a string value and nothing else. In Redis, the string key can be mapped to more than just a simple string.
Being a data structure server, we can also refer to the data types as data structures. We can use these more complex data structures to store multiple values in a key at once. Let's look at these types at a high level. We'll explore each type in detail in subsequent sections.
Binary-safe Strings
The most basic kind of Redis value. Being "binary-safe" means that the string can contain any type of data represented as a string: PNG images or serialized objects, for example.
Lists
In essence, Redis Lists are linked lists. They are collections of string elements that are sorted based on the order that they were inserted.
Sets
They represent collections of unique and unsorted string elements.
Sorted Sets
Like Sets, they represent a collection of unique string elements; however, each string element is linked to a floating number value, referred to as the elementscore. When querying the Sorted Set, the elements are always taken sorted by their score, which enables us to consistently present a range of data from the Set.
Hashes
These are maps made up of string fields linked to string values.
Bit arrays
Also known as bitmaps. They let us handle string values as if they were an array of bits.
HyperLogLogs
A probabilistic data structure used to estimate thecardinalityof a set, which is a measure of the "number of elements of the set."
We have already covered Strings during the "Write, Read, Update, and Delete Data in Redis" section. For the rest of this tutorial, we are going to focus on all the Redis types except bitmaps and hyperloglogs. We'll visit those on a future post handling an advanced Redis use case.
Lists
A List is a sequence of ordered elements. For example,1245690193is a List of numbers. In Redis, it's important to note that Lists are implemented aslinked lists. This has some important implications regarding performance. It is fast to add elements to the head and tail of the List but it's slower to search for elements within the List as we do not have indexed access to the elements (like we do in an array).
A List is created by using a Redis command that pushes data followed by a key name. There are two commands that we can use:RPUSHandLPUSH. If the key doesn't exist, these commands will return a new List with the passed arguments as elements. If the key already exists or it is not a List, an error is returned.
RPUSH
RPUSHinserts a new element at the end of the List (at the tail):
RPUSH key value [value ...]
Let's create anengineerskey that represents a List:
Each time we insert an element, Redis replies with the length of the List after that insertion. We would expect theuserslist to resemble this:
Alice Bob Carmen
How can we verify that? We can use theLRANGEcommand.
LRANGE
LRANGEreturns a subset of the List based on a specified start and stop index. Although these indexes are zero-based, they are no the same as array indexes. Given a full List, they simply indicate where to partition the List: make a slice from here (start) to here (stop):
LRANGE key start stop
To see the full List, we can use a neat trick: go from0to the element just before it,-1.
LRANGE engineers 0 -1
Redis returns:
1) "Alice"
2) "Bob"
3) "Carmen"
The index-1will always represent the last element in the List.
To get the first two elements ofengineerswe can issue the following command:
LRANGE engineers 0 1
LPUSH
LPUSHbehaves the same asRPUSHexcept that it inserts the element at the front of the List (at the header):
LPUSH key value [value ...]
Let's insertDanielat the front of theengineerslist:
LPUSH engineers "Daniel"
// 4
We now have four engineers. Let's verify that the order is correct:
LRANGE engineers 0 -1
Redis replies with:
1) "Daniel"
2) "Alice"
3) "Bob"
4) "Carmen"
It's the same List we had before but with"Daniel"as the first element, which is exactly what was expected.
Multiple Element Insertions
We saw in the signatures ofRPUSHandLPUSHthat we can insert more than one element through each command. Let's see that in action.
Based on our existingengineerslist, let's issue this command:
RPUSH engineers "Eve" "Francis" "Gary"
// 7
Since we are inserting them at the end of the List, we expect these three new elements to show up in the same order in which they are listed as arguments. Let's verify:
When listing multiple arguments forLPUSHandRPUSH, Redis inserts the elements one by one, thus,"Hugo","Ivan", and"Jess"appear in the reverse order from which they were listed as arguments.
LLEN
We can find the length of a List at any time by using theLLENcommand:
LLEN key
Let's verify that the length ofengineersis indeed10:
LLEN engineers
Redis replies with(integer)10. Perfect.
Removing Elements from a Redis List
Similar to how we can "pop" elements in arrays, we can pop an element from the head or the tail of a Redis List.
LPOPremoves and returns the first element of the List:
LPOP key
We can use it to remove"Jess", the first element, from the List:
LPOP engineers
Redis indeed replies with"Jess"to indicate it is the element that was removed.
RPOPremoves and returns the last element of the List:
RPOP key
It's time to say goodbye to"Gary", the last element of the List:
RPOP engineers
The reply from Redis is"Gary".
It's very useful to be able to get the element that was removed from the List as we may want to do something special with it.
In Redis, a Set is similar to a List except that it doesn't keep any specific order for its elements and each element must be unique.
SADD
We create a Set by using theSADDcommand that adds the specified members to the key:
SADD key member [member ...]
Specified members that are already part of the Set are ignored. If the key doesn't exist, a new Set is created and the unique specified members are added. If the key already exists or it is not a Set, an error is returned.
Let's create alanguagesset:
SADD languages "english"
// 1
SADD languages "spanish"
// 1
SADD languages "french"
// 1
In this case, on each member addition Redis returns the number of members that were added with theSADDcommand, not the size of the Set. Let's see this in action:
SADD languages "chinese" "japanese" "german"
// 3
SADD languages "english"
// 0
The first command returned3as we were adding three unique members to the Set. The second command returned0as"english"was already a member of the Set.
SREM
We can remove members from a Set by using theSREMcommand:
SREM key member [member ...]
We can remove one or more members at the same time:
SREM languages "english" "french"
// 2
SREM languages "german"
// 0
SREMreturns the number of members that were removed.
SISMEMBER
To verify that a member is part of a Set, we can use theSISMEMBERcommand:
SISMEMBER key member
If the member is part of the Set, this command returns1; otherwise, it returns0:
SISMEMBER languages "spanish"
// 1
SISMEMBER languages "german"
// 0
Since we removed"german"in the last section, we get0.
SMEMBERS
To show all the members that exist in a Set, we can use theSMEMBERScommand:
SMEMBERS key
Let's see what language values we currently have in thelanguagesset:
Something really powerful that we can do with Sets very fast is to combine them using theSUNIONcommand:
SUNION key [key ...]
Each argument toSUNIONrepresents a Set that we can merge into a larger Set. It is important to notice that any overlapping members will be listed once.
To see this in action, let's first create anancient-languagesset:
If we pass toSUNIONa key that doesn't exist, it considers that key to be an empty set (a set that has nothing in it).
Hashes
In Redis, a Hash is a data structure that maps a string key with field-value pairs. Thus, Hashes are useful to represent objects. Theykeyis the name of the Hash and thevaluerepresents a sequence offield-name field-valueentries. We could describe acomputerobject as follows:
computer name "MacBook Pro" year 2015 disk 512 ram 16
The "properties" of the object are defined as sequences of "property name" and "property value" after the name of the object,computer. Recall that Redis is all about sequential strings so we have to be careful when creating these string objects that we use the proper string sequencing to define our objects correctly.
To manipulate Hashes, we use commands that are similar to what we used with strings, after all, they are strings.
Writing and Reading Hash Data
HSET
The commandHSETsetsfieldin the Hash tovalue. Ifkeydoes not exist, a new key storing a hash is created. Iffieldalready exists in the hash, it is overwritten.
HSET key field value
Let's create thecomputerhash:
HSET computer name "MacBook Pro"
// 1
HSET computer year 2015
// 1
HSET computer disk 512
// 1
HSET computer ram 16
// 1
For eachHSETcommand, Redis replies with an integer as follows:
1iffieldis a new field in the hash and value was set.
0iffieldalready exists in the hash and the value was updated.
Let's update the value of theyearfield to2018:
HSET computer year 2018
// 0
HGET
HGETreturns the value associated withfieldin a Hash:
HGET key field
Let's verify that we are getting2018as the value ofyearinstead of2015:
HGET computer year
Redis replies with"2018". It checks out fine.
HGETALL
A fast way to get all the fields with their values from the hash is to useHGETALL:
HGETALLreplies with an empty list when the providedkeyargument doesn't exist.
HMSET
We can also set multiple fields at once usingHMSET:
HMSET key field value [field value ...]
Let's create atablethash with it:
HMSET tablet name "iPad" year 2016 disk 64 ram 4
HMSETreturnsOKto let us know thetablethash was created successfully.
HMGET
What if we want to get just two fields? We useHMGETto specify from which fields in the hash we want to get a value:
HMGET key field [field ...]
Let's get thediskandramfields of thetablethash:
HMGET tablet disk ram
Effectively we get the values ofdiskandramas replies:
1) "64"
2) "4"
That's pretty much the gist of using Hashes in Redis. You may explore thefull list of Hash commandsand try them out.
Sorted Sets
Introduced in Redis 1.2, a Sorted Set is, in essence, a Set: it containsunique, non-repeating string members. However, while members of a Set are not ordered (Redis is free toreturn the elements in any order at every callof a Set), each member of a Sorted Set is linked to a floating point value called thescorewhich is used by Redis to determine the order of the Sorted Set members. Since, every element of a Sorted Set is mapped to a value, it also has an architecture similar to Hash.
In Redis, a Sorted Set could be seen as a hybrid of a Set and a Hash.
How is the order of members of a Sorted Set determined? As stated in theRedis documentation:
If A and B are two members with a different score, then A > B if A.score is > B.score.
If A and B have exactly the same score, then A > B if the A string is lexicographically greater than the B string.A and B strings can't be equal since Sorted Sets only have unique elements.
Some of the commands that we use to interact with Sorted Sets are similar to the commands we used with Sets: we replace theSin the Set command and replace it with aZ. For example,SADD=>ZADD. However, we have commands that are unique to both. Let's check them out.
ZADD
UsingZADDadds all the specified members with specified scores to the Sorted Set:
ZADD key [NX|XX] [CH] [INCR] score member [score member ...]
As with Sets, ifkeydoes not exist, a new Sorted Set with the specified members as only members is created. If thekeyexists but does not hold a Sorted Set, an error is returned.
XX: Only update members that already exist. Never add members.
NX: Don't update already existing members. Always add new members.
CH: Modify the return value from the number of new members added, to the total number of members changed (CH is an abbreviation of changed). Changed members are new members addedandmembers already existing for which the score was updated. So members specified in the command line having the same score as they had in the past are not counted.
INCR: When this option is specified ZADD acts likeZINCRBY. Only one score-members pair can be specified in this mode.
It's good to know that these optional arguments are there and what they do, but for this introduction, we are going to focus on adding members without using any of them, but feel free to explore them! In future posts, we are going to revisit them in more complex use cases!
Let's create a Sorted Set to store Help Desk Support tickets. Support tickets are meant to be unique but also need to be sorted, hence, this data structure is a great choice:
ZADDreturns a count the number of new elements added. In the commands above, we used the position of the ticket in a queue as the score value followed by the ticket number (all fictional).
ZRANGE
We'd like now to see how our Sorted Set looks. With Sets, we usedSMEMBERSto list the unordered members. With Sorted Sets, we use a command that is more in tune with what we used with Lists, a command that shows us a range of elements.
ZRANGEreturns the specified range of members in the Sorted Set:
ZRANGE key start stop [WITHSCORES]
It behaves very similarly toLRANGEfor Lists. We can use it to get a subset of the Sorted Set. To get the full Sorted Set, we can use the0-1range again:
ZRANGE tickets 0 -1
Redis replies with:
1) "HELP004"
2) "HELP204"
3) "HELP330"
We can passZRANGEtheWITHSCORESargument to also include the score of each member:
Notice how thememberand thescoreare listed in sequence and not next to each other. As we can see, the members are stored inticketsin ascending order based on their score.
Using Redis as a Session Store
The most relevant use of Redis in the authentication and authorization workflows of a web application is to serve as a session store.
As recognized byAmazon Web Services, the in-memory architecture of Redis provides developers with high availability and persistence that makes it a popular choice to store and manage session data for internet-scale applications. Its lightning-fast performance provides us with the super low latency, optimal scale, and resiliency that we need to manage session data such as user profiles, user settings, session state, and credential management.
Roshan Kumar, fromRedis Labs, explains on his"Cache vs. Session Store"article that a session-oriented web application starts a session when the user logs in. The session is active until the user logs out or the session times out. During the session lifecycle, the web application stores all session-related data in the main memory (RAM) or in a session store that doesn't lose the data when the application goes down. This session store can be implemented using Redis that, despite being an in-memory store, is able to persist data bywriting transaction logs sequentially in the disk.
Roshan further explains that session stores rely on reading and writing data to the in-memory database. The session store data isn’t temporary and it becomes the only source of truth when the session is live. For that reason, the session store needs to meet the "data durability requirements of a true database."
"According to @RedisLabs, a session store requires high availability and durability to support transactional data and uninterrupted user engagement. You can achieve that easily using #Redis."
Redis is a powerful, nimble, and flexible database that can speed up your architecture. It has a lot to offer including caching, data replication, pub/sub messaging systems, session storage, and much more. Redis has amultitude of clientsthat cover all of the popular programming languages. I hope that you try it out whenever you have a use case that fits its value propositions