While making some parts of this website and some authoring other tools I needed to dabble a little with the OSRS cache. For example, to help build the OSRSBox Item Database project (osrsbox-db), I needed to extract all items from the OSRS cache. Along the way, I learned a lot about the cache structure, the type of information it contains and various tools that are available to help parse the information in the cache. This page documents some of the more useful information I found to hopefully help someone else who is interested.
This post starts with a brief discussion about the OSRS cache, then outlines how to use the open source RuneLite project to extract item definitions and npc definitions from the cache.
The OSRS Cache structure is pretty much the same as the very well know #317 Runescape 2 cache format. All cache files are stored in the home directory of the user. In Microsoft Windows (Vista onwards), the cache files are stored in the following directory:
Similarly, on Linux-based systems the cache files are stored in the following directory:
Inside the cache directory, there are a number of files, but there are only two primary file types/formats. These file types are easily identified by their corresponding file extension, namely:
The following sections discuss each of these file type/formats in further detail.
The primary data file is named
main_file_cache.dat2. This file contains the entire OSRS cache including information about items, npcs, images, sounds, models, and animations. However, the file is stored in a complex structure and you cannot just open the
main_file_cache.dat2 file and view images or extract music. The file is made up of various blocks of data, some are compressed, some are not, and the information is not trivial to extract. To extract anything from the
main_file_cache.dat2 file you must know the location of the data blocks. This information is provided by index files.
As the name suggests, the index file is an index for the
main_file_cache.dat2 file. In OSRS there are currently 17 index files named consecutively from
main_file_cache.idx16, as well as one index file named
main_file_cache.idx255. The table below documents each index file, and the data that the index file points to.
|Index file name||Content|
|main_file_cache.idx2||Config (See below)|
|main_file_cache.idx4||Sound Effects 1|
|main_file_cache.idx14||Sound Effects 2|
|main_file_cache.idx15||Sound Effects 3|
The config file, or
main_file_cache.idx2 contains highly useful information.
|Index 2 block||Content|
|12||Animation Definition (sequence)|
|13||Animated Graphic Definition (Spot Animation/GFX)|
RuneLite is a suite of open source tools for Old School RuneScape. The project provides a free, open-source and super fast client for Old School RuneScape. In addition, the creators also provide a cache extraction tool and used to provide a client deobfuscator. This documentation focuses on the cache tool for extracting information from the OSRS cache, and dabbles a little later with the deobfuscator (using an old version from the RuneLite project) for extracting cache version information. The RuneLite homepage provides a summary of the project and precompiled downloads for the client. The RuneLite GitHub project page is the official repository to get the source code for the project.
The RuneLite client comes as a pre-compiled download (a
.jar file). However, the other tools that are part of the suite of tools do not come precompiled. Therefore, this section documents how to compile the tools. Although this is a pretty straight-forward process, it may be difficult for someone without much programming experience.
The following instructions were performed on Microsoft Windows 10 64-bit. However, the instructions should be suitable for other Microsoft Windows operating systems and also adaptable to other operating systems such as Linux. The following software is required: 1) Git; 2) Netbeans; 3) Java.
Now, clone the RuneLite repository using Git Bash. For those new to git, you need to open Git Bash using the Windows Explorer context menu. In Windows Explorer, browse to a folder where you want to down the RuneLite repository, right-click a white-space area in Windows Explorer, and select Git Bash. Then enter the following command into the Git Bash command prompt to download the source code from the RuneLite GitHub repository:
Excellent, we have everything ready to go.
Open the NetBeans IDE from the Windows Start Menu. We will start by making a couple of configurations to Netbeans to make life easier. Firstly, we will disable testing, mainly because it takes a very long time to compile the project with additional tests. Perform the following tasks:
Now, open the RuneLite repository in the NetBeans IDE:
Before we can build the actual cache tool, we need to modify the source code a little. Since RuneLite is a suite of tools, we need to configure the Maven project to only build the Cache tool. Using the navigation menu, open the following file:
You need to add a line in the
pom.xml file to specify the main class. With the main class, the tool cannot be run correctly. You need to add the following entry:
The entry needs to be in the
<plugin> section, specifically in the
maven-assembly-plugin section. Inside this XML entry is a
configuration tag, then a
archive tag. Basically, you need to add a
manifest tag that states that the resultant build file (
.jar), should use the Cache main class as the program entry point. Without this addition, you will get a
no main manifest attribute error when running the
.jar file. The following snippet displays a section of the
pom.xml file, and what it should look like after the entry is added.
Great! Now you can build the cache tool. Perform the following tasks:
This will compile the Cache tool and produce a
.jar file that can be executed. The build process may take a while for the first build because a collection of dependencies need to be downloaded. The
.jar file for the cache tool is available in the following directory:
Two different versions will be compiled:
As illustrated by the naming conventions, one has a
.jar build with all project dependencies. The other is a standalone
.jar file without dependencies. It is recommended to the use the
cache-1.4.11-SNAPSHOT-jar-with-dependencies.jar file to avoid any dependency issues.
The cache tool has support for extracting the following types of data from the OSRS cache:
You can run the newly build cache tool by using the Windows Command environment. The following steps should help:
cmdand left-click to open
The cache tool has the following general syntax:
Where each of the inputs are explained below:
<cache-location>is the location of the OSRS cache. This is usually in the home directory; for example:
C:\Users\<username>\jagexcache\oldschool\LIVE\. In the following command examples, the username is:
<cache-extraction-option>is the type of information that you want to extract. Valid options are
<output-directory>is the directory to save the output. Sometimes this will be created automatically by the tool, and sometimes you will have to manually create it. In the following command examples, as output is saved to the
Desktopfolder for the user named
Examples are helpful for this type of tool, where the command can be complex. The following five snippets provide examples of how to run the RuneLite cache tool.
You can also run the cache tool directly in Netbeans. However, to achieve this you must provide command line arguments within the NetBeans IDE. This is complex to set up, but simple after it is configured. To achieve this, in the bottom-right pane (the output/debugging pane) there is a little icon that looks like a fast-forward button - colored (yellow). There are two icons which look the same, it is the bottom one! When hovered over, it will read: Re-run with different parameters. This button allows us to enter command line arguments when running the program. The first line needs to have the arguments added to it. In the original format, the configuration has the following line:
If you want to Run the cache tool directly in Netbeans you need to add additional arguments. For example, to dump items you can use the following arguments:
The information I wanted to extract from the OSRS Cache was the item definitions. I needed this information to get an up-to-date and complete list of all items in OSRS. The code snippet below shows one item definition that has been extracted from the cache. The specific item has the ID number of
22111 and is the
As you can see there is a variety of information - some useful, some not. The usefulness of the information really depends on what information you are after!