Links

11211 - Pentesting Memcache

Protocol Information

Memcached (pronunciation: mem-cashed, mem-cash-dee) is a general-purpose distributed memory caching system. It is often used to speed up dynamic database-driven websites by caching data and objects in RAM to reduce the number of times an external data source (such as a database or API) must be read. (From wikipedia) Although Memcached supports SASL, most instances are exposed without authentication.
Default port: 11211
PORT STATE SERVICE
11211/tcp open unknown

Enumeration

Manual

To ex-filtrate all the information saved inside a memcache instance you need to:
  1. 1.
    Find slabs with active items
  2. 2.
    Get the key names of the slabs detected before
  3. 3.
    Ex-filtrate the saved data by getting the key names
Remember that this service is just a cache, so data may be appearing and disappearing.
echo "version" | nc -vn -w 1 <IP> 11211 #Get version
echo "stats" | nc -vn -w 1 <IP> 11211 #Get status
echo "stats slabs" | nc -vn -w 1 <IP> 11211 #Get slabs
echo "stats items" | nc -vn -w 1 <IP> 11211 #Get items of slabs with info
echo "stats cachedump <number> 0" | nc -vn -w 1 <IP> 11211 #Get key names (the 0 is for unlimited output size)
echo "get <item_name>" | nc -vn -w 1 <IP> 11211 #Get saved info
​
#This php will just dump the keys, you need to use "get <item_name> later"
sudo apt-get install php-memcached
php -r '$c = new Memcached(); $c->addServer("localhost", 11211); var_dump( $c->getAllKeys() );'

Manual2

sudo apt install libmemcached-tools
memcstat --servers=127.0.0.1 #Get stats
memcdump --servers=127.0.0.1 #Get all items
memccat --servers=127.0.0.1 <item1> <item2> <item3> #Get info inside the item(s)

Automatic

nmap -n -sV --script memcached-info -p 11211 <IP> #Just gather info
msf > use auxiliary/gather/memcached_extractor #Extracts saved data
msf > use auxiliary/scanner/memcached/memcached_amp #Check is UDP DDoS amplification attack is possible

Dumping Memcache Keys

If your memcached version is above 1.4.31, read next section for advanced method of dumping keys.
The memcache protocol provides commands to peek into the data that is organized by slabs (categories of data of a given size range). There are some significant limitations though:
  1. 1.
    You can only dump keys per slab class (keys with roughly the same content size)
  2. 2.
    You can only dump one page per slab class (1MB of data)
  3. 3.
    This is an unofficial feature that might be removed anytime.​
The second limitation is probably the hardest because 1MB of several gigabytes is almost nothing. Still it can be useful to watch how you use a subset of your keys. But this might depend on your use case. If you don’t care about the technical details just skip to the tools section to learn about what tools allow you to easily dump everything. Alternatively follow the following guide and try the commands using telnet against your memcached setup. How it Works First you need to know how memcache organizes its memory. If you start memcache with option “-vv” you see the slab classes it creates. For example
$ memcached -vv
slab class 1: chunk size 96 perslab 10922
slab class 2: chunk size 120 perslab 8738
slab class 3: chunk size 152 perslab 6898
slab class 4: chunk size 192 perslab 5461
[...]
In the configuration printed above memcache will keep fit 6898 pieces of data between 121 and 152 byte in a single slab of 1MB size (6898*152). All slabs are sized as 1MB per default. Use the following command to print all currently existing slabs:
stats slabs
If you’ve added a single key to an empty memcached 1.4.13 with
set mykey 0 60 1
1
STORED
you’ll now see the following result for the “stats slabs” command:
stats slabs
STAT 1:chunk_size 96
STAT 1:chunks_per_page 10922
STAT 1:total_pages 1
STAT 1:total_chunks 10922
STAT 1:used_chunks 1
STAT 1:free_chunks 0
STAT 1:free_chunks_end 10921
STAT 1:mem_requested 71
STAT 1:get_hits 0
STAT 1:cmd_set 2
STAT 1:delete_hits 0
STAT 1:incr_hits 0
STAT 1:decr_hits 0
STAT 1:cas_hits 0
STAT 1:cas_badval 0
STAT 1:touch_hits 0
STAT active_slabs 1
STAT total_malloced 1048512
END
The example shows that we have only one active slab type #1. Our key being just one byte large fits into this as the smallest possible chunk size. The slab statistics show that currently on one page of the slab class exists and that only one chunk is used. Most importantly it shows a counter for each write operation (set, incr, decr, cas, touch) and one for gets. Using those you can determine a hit ratio! You can also fetch another set of infos using “stats items” with interesting counters concerning evictions and out of memory counters.
stats items
STAT items:1:number 1
STAT items:1:age 4
STAT items:1:evicted 0
STAT items:1:evicted_nonzero 0
STAT items:1:evicted_time 0
STAT items:1:outofmemory 0
STAT items:1:tailrepairs 0
STAT items:1:reclaimed 0
STAT items:1:expired_unfetched 0
STAT items:1:evicted_unfetched 0
END
What We Can Guess Already… Given the statistics infos per slabs class we can already guess a lot of thing about the application behaviour:
  1. 1.
    How is the cache ratio for different content sizes?
    • How good is the caching of large HTML chunks?
  2. 2.
    How much memory do we spend on different content sizes?
    • How much do we spend on simple numeric counters?
    • How much do we spend on our session data?
    • How much do we spend on large HTML chunks?
  3. 3.
    How many large objects can we cache at all?
Of course to answer the questions you need to know about the cache objects of your application. Now: How to Dump Keys? Keys can be dumped per slabs class using the “stats cachedump” command.
stats cachedump <slab class> <number of items to dump>
To dump our single key in class #1 run
stats cachedump 1 1000
ITEM mykey [1 b; 1350677968 s]
END
The “cachedump” returns one item per line. The first number in the braces gives the size in bytes, the second the timestamp of the creation. Given the key name you can now also dump its value using
get mykey
VALUE mykey 0 1
1
END
This is it: iterate over all slabs classes you want, extract the key names and if need dump there contents.

DUMPING MEMCACHE KEYS (VER 1.4.31+)

In memcache version 1.4.31 and above there is a new command for dumping memory keys in non-blocking mode (read https://github.com/memcached/memcached/wiki/ReleaseNotes1431 ). This method is safe to run in production. The output is not consistent, but good enough for finding keys, their exact expiration time (EXP) and last accessed time (LA). Because of huge output generated, it’s recommended to use ‘nc’ command. Examples:
echo 'lru_crawler metadump all' | nc 127.0.0.1 11211 | head -1
key=0dLLX%253Amemcache_test_key exp=1590718787 la=1590718487 cas=2238881166 fetch=yes
​
echo 'lru_crawler metadump all' | nc 127.0.0.1 11211 | grep ee6ba58566e234ccbbce13f9a24f9a28
key=VQRFX%253Aee6ba58566e234ccbbce13f9a24f9a28 exp=-1 la=1590386157 cas=1776204003 fetch=yes
key=0dLLX%253Aee6ba58566e234ccbbce13f9a24f9a28 exp=-1 la=1590712292 cas=2225524871 fetch=yes
EXP=-1 means the item never expires EXP=1590718787 (Fri May 29 02:19:47 GMT 2020) keeps the unix timestamp when the item should expire LA=1590712292 (Mon May 25 05:55:57 GMT 2020) keeps the unix timestamp when the item was last accessed

DUMPING TOOLS

There are different dumping tools sometimes just scripts out there that help you with printing memcache keys:
Programming Languages
Tools
Functionality
Text
Text
PHP
​simple script​
Prints key names.
​
​
Perl
​simple script​
Prints keys and values
​
​
Ruby
​simple script​
Prints key names.
​
​
Perl
​memdump​
Tool in CPAN module
ached/)
PHP
​memcache.php​
Memcache Monitoring GUI that also allows dumping keys
​
​
libmemcached
​peep​
Does freeze your memcached process!!! Be careful when using this in production. Still using it you can workaround the 1MB limitation and really dump all keys.
​
​

Troubleshooting

1MB Data Limit

Note that prio to memcached 1.4 you cannot store objects larger than 1MB due to the default maximum slab size.

Never Set a Timeout > 30 Days!

If you try to “set” or “add” a key with a timeout bigger than the allowed maximum you might not get what you expect because memcached then treats the value as a Unix timestamp. Also if the timestamp is in the past it will do nothing at all. Your command will silently fail.
So if you want to use the maximum lifetime specify 2592000. Example:
set my_key 0 2592000 1
1

Disappearing Keys on Overflow

Despite the documentation saying something about wrapping around 64bit overflowing a value using “incr” causes the value to disappear. It needs to be created using “add”/”set” again.

Replication

memcached itself does not support replication. If you really need it you need to use 3rd party solutions:
  • ​repcached: Multi-master async replication (memcached 1.2 patch set)
  • ​Couchbase memcached interface: Use CouchBase as memcached drop-in
  • ​yrmcds: memcached compatible Master-Slave key value store
  • ​twemproxy (aka nutcracker): proxy with memcached support

Commands Cheat-Sheet

Shodan

  • port:11211 "STAT pid"
  • "STAT pid"

References