Sims 4 Tuning 101: A Deep Dive into how Tuning is Generated from Python — Part 3

This is a continuation of my last article about defining XML tuning in Python. In this section, I will focus on how the XML tuning gets loaded into the game as Python objects.

This article will assume you have some understanding of XML tuning and Python scripting as it pertains to the Sims 4 codebase, but I will try my best to explain things in a way that doesn’t force you to know Python in depth.

Section 3: Loading Tunables

My last two articles focused on how the XML tuning is organised and how it is defined in Python. In this article, I am going to focus again on Python, but this time on how it is loaded into the game. As mentioned before, it is important to understand that the way you define the tunables is not the way they are stored and used by the game: you will never see a TunableList be treated like a TunableList object in the game’s code at the point where the code is loaded, and likewise, none of the other tunables will have any idea of the data structures they came from.

Knowing how the game loads and stores tuning is important for those who wish to modify the code through script, as knowing what data structures are in use is imperative in being able to alter them.

If you’re ambitious enough to attempt your own tuning, which I will cover next, then knowing both the before and the after when it comes to the tunables will be particularly pertinent to you.

Python Data Structures

Before I cover how XML tuning gets loaded into the game, I’m going to take a brief detour to explain some of the basics of data structures, and in particular, data structures in Python. For a more in depth, I suggest you read this article here, but I will try to explain the most important basics that you will need to know here.

What Is a Data Structure?

The tl;dr on what a data structure is, is basically they are a way to describe and organise data, usually to serve some purpose. For example, some data you want organise in order; some data the order doesn’t matter and it’s more important that the values you have are just unique; some data you want to be able to look up fast; etc.

For the purposes of this tutorial, and for the the XML tunables, we will only need to look at a few data structures: sets, tuples and lists, and dictionaries (dict in Python terms).

Sets

I already explained sets in the previous two articles, so I won’t dwell on them too much here.

In Python, sets cannot be indexed, but they can be iterated over. There are two types of sets, “normal sets” and frozensets, which are like normal set objects, except that they are immutable.

Tuples and Lists

Tuples and Lists are pretty much the same data structure in Python, but differ in one very important way: tuples are immutable, lists are not.

Lists and tuples allow for collecting and storing data of any type, and they can be accessed by the index operator (the square brackets, []), using the index (thus the name) of the item in the list or tuple. If the index is outside the length of the list or tuple, you will get an IndexError Exception.

Dictionaries

Dictionaries, also called associative arrays and mappings, are data structures that are organised by their keys which refer to values. In Python, the keys must be hashable, as I explained in the last article, but the values can be anything.

To get a value from a dictionary, you can also use the index operator and look up the desired key. If the key does not exist, you will get a KeyError Exception.

The Sims 4 codebase comes with a special type of dictionary called a frozendict, which is like a normal dict object, except it is immutable.

How This Relates to the XML Tuning

Different tunables will be loaded by the game into different data structures. However, one important thing to keep in mind is that all tunables are loaded into the game as immutable, meaning that if you want to modify their values, you must first convert them into a mutable form and then convert back.

The <E> Tag

While the <E> tag can be a little complicated on paper, when it comes to its representation in Python, it is loaded simply as an Enum (in particular, the Enum that it was defined to belong to in the Python). Since Enums are essentially “fancy integers”, you can do to them pretty much anything you can do with other numerical data types in Python. That said, you will likely only need to be comparing them against other values from the same Enum.

The <T> Tag

Slightly more complicated is the <T> tag since how it gets loaded depends on how it’s defined.

For primitive <T>s — those that are either of type bool, str, or int/float, they conveniently get loaded into the Python as the data types they are defined as. So, if you have a Tunable of type str, it will be a str object with the value you gave it in the XML. If the tunable is not given a value explicitly in the XML, it is given the default value provided to it in the Python (often None for str, False for bool, and 0 for int/float — but you shouldn’t assume the defaults for tuning you did not create: always consult the TDESCs when in doubt).

For resource <T>s, they are loaded as basically “fancy integers” and generally need to be sent to the C++ to really mean anything. In the case for Strings, we only get access to their ID and can pass to them tokens. It is the C++ that is responsible for taking the String and turning it into the words we see on the screen. These resources are, to use a programming term, basically opaque pointers: we don’t know where they “point” to, or even really what, we simply have the pointer and can pass it around to things that know how to use it better.

Reference <T>s get loaded by the game directly as either a None if unable to be found (and the TunableReference is pack safe), or the actual Tuning class itself if it can be found. This is the true power of TunableReferences, especially if you’re making custom tuning: no more needing to load the code yourself! Because they are the Tuning class itself, you can modify properties about it directly* — wanna change the category of an Interaction? Go right ahead. Need to add some _loot_on_additions to a Buff? No problem! The names of the properties match the names in the XML**.

The <L> Tag

The <L> tag is unfortunately a bunch of things all at once, and how it’s loaded in the game depends on what Tunable it came from. In this section, I will describe each of the different Tunables that get represented as an <L> and how they are represented as Python objects.

TunableLists

TunableLists get loaded into the game as tuples, meaning that to modify them, you will need to either convert them into a list or you’ll have to use an operator that returns a new one (such as +=).

To look at how this works, let’s take a look at an example.

If we have this XML:

Which can be defined by this Python:

Would get loaded by the game as though it were defined as the following:

TunableSets

TunableSets get loaded into the game as frozensets, meaning that to modify them, you will need to either convert them into a set or you’ll have to use an function that returns a new one (such as union()).

Using the same <L> as above, which would be defined by this Python instead:

Would be loaded into the game as so:

TunableMappings

TunableMappings get loaded into the game as frozendicts. To modify them, you will have to convert them into a dict object and then convert them back after you are done.

For tunable mappings, the key/value pairs in the XML tuning will correspond to the keys and values in the frozen dictionary object when loaded by the game.

Going back to the example of a TunableMapping before, this will be loaded into the game as though it were created with the following Python code directly:

Note that when the mapping gets loaded into the game, the names of the keys and values used to construct the mapping do not matter.

The <U> Tag

The <U> tag is either loaded as an ImmutableSlots object or, in the case of TunableFactories, as an instance of a specific class. For this tutorial, I am going to focus on the ImmutableSlots case and come back at a later time to TunableFactories and how to deal with them.

The properties of the ImmutableSlots class returned will be the same as those defined in the Python/as in the XML, but, as the name might suggest, you are unable to modify them and can only access them.

To get around this problem, there is a method on ImmutableSlots objects called clone_with_overrides() which allows for you to create a new ImmutableSlots object from the old one with its properties overridden as desired.

Because of this, modifying complex objects with nested ImmutableSlots objects may become tedious, and unfortunately, there is not much around this fact.

Let’s take an look at an example, which will use a lot of what we already know:

Here’s some code that could be used to define a Periodic Table of Elements (more specifically, a TunableList of TunableTuples, where the TunableTuple defines the element).

If we used the following XML:

We would in Python receive an object with the following data:

Note that this is a somewhat approximate representation of the data; if you were to print to a file the representation of the Tuning above in your own game with Python, you’d get something similar but not quite the same (the representation of Strings looks a little different, as do enumerations, and ImmutableSlots themselves). However, for illustrative purposes, the above should give you a rough idea of how they are loaded into the game.

If we wanted to modify the third element in the list, Lithium, to have, say, a different atomic mass, we would do it like so (assuming the list was stored in a variable called periodic_table):

One thing to notice here is that, because clone_with_overrides returns a new instance of the ImmutableSlots class, we must replace the old one. However, since periodic_table is a tuple, we have to convert it into a list first so we can modify it.

This sort of process will appear time and time again when doing script modifications to the game code, so it’s important to get a general feel for how this is done.

The <V> Tag

The <V> tag is difficult to explain because it is so context dependent: it gets loaded by the game as whatever tunable variant was chosen. However, as I explained before, at load-time, you have no way of directly knowing which variant was chosen without some sort of test. Because of this, if you do not know ahead of time which variant was selected, you may need to be a little creative.

Depending on if the variant is given a default value, a variant that is not defined may either by loaded by the game as a None or as the default-constructed version of the default value. This can be useful when constructing your own tuning, as it allows for you to save needing to enter values for every tunable item.

In the next part, I will talk about some strategies that can be used with this knowledge in mind to modify in-game tuning through script injection.

Footnotes

1. * Depends on the field and the Tunable. If the Tunable has SimData (like the name and description of Buffs), then you cannot modify the field with Python and expect the change to be reflected in the game’s UI, or even be reflected in the game at all. Complex tunables also may need to be dealt with more surgically (especially TunableFactories) in order to be modified.
2. ** This is the case at least for the tunable fields of Tunables (like Interactions and Buffs, etc.). Accessing the properties inside the tunable fields may be a little more difficult and largely depends on the type of tunable. TunableFactories for example may not “match” the XML used to represent them very well.