Godot's import process and you - Part 1: Peeking behind the scenes

Have you ever wondered what's up with all these files with the .import extension in your Godot project? Whether you should commit them to version control? And what is all that stuff inside the .godot folder in the project's root directory? Or you have a rough idea but would like to know more details? If so you have come to the right article!

A gourd example

Most games need assets, be it images, sounds, 3D models, or even custom file formats. But the form those assets are provided in is often not fit for the needs of a game. For example, you may draw a texture in Krita and export it as a PNG, which is a compressed file format. To display that texture, it must first be decompressed on the CPU using the PNG algorithm and then sent to the GPU. But raw textures can take up a lot of VRAM, the memory available to your graphics card. Let's say your game needs a 4K gourd texture. With four bytes per pixel, that's 4096 * 4096 * 4 B = 64 MB! This quickly adds up once you give your tomatoes and potatoes the same treatment.

To reduce VRAM usage one can use a compression algorithm that works entirely on the GPU, such as UASTC. These algorithms can't compress images as much as PNG or JPEG, which is why you don't see them being used to store files on disk, but in return they make it possible to store compressed textures in VRAM and decode them very quickly on the fly whenever the data is accessed.

So let's say you store your gourd texture as a PNG. When a player visits the farmers market, that file is loaded from disk, decompressed on the CPU using the PNG algorithm, then the raw data gets compressed using a GPU compression algorithm and sent to the graphics card. All this adds to the time the game needs to load, delaying the high adrenaline produce bargain hunting your players seek. To speed things up, we should do as much work ahead of time as possible and not unnecessarily repeat steps during runtime. Which means that when you ship your game, you wouldn't include the PNG texture at all, but use a GPU compressed version of it. (To be precise, you would probably want to use an intermediate file format such as Basis Universal which can quickly be transcoded to multiple different GPU texture formats).

What's so .important about this anyway?

So you see that there is a need for processing assets before they can be optimally used by the engine. But your image editing software doesn't have the option to export files using GPU texture formats, or maybe you just want to quickly throw in a JPEG file you found on the internet for prototyping. That's why game engines automate the process, conveniently letting you work with whatever formats you are comfortable with and importing them to something the game can use behind the scenes. But the right way to convert an asset depends on how it is meant to be used, which the engine can't figure out on its own. That's where .import files come in.

If you have worked with Godot and looked into project folder using a file manager then you have seen them: every asset is accompanied by another file with the same name but suffixed by .import. These are simple, human readable text files similar to .tres files for resources. These contain all the data needed for proper conversion, such as which GPU compression algorithm to use, whether mipmaps should be generated, etc. You can edit them manually in a text editor, but the primary way of modifying them is via the Godot editor.

To do so, you select the relevant file in the file system panel, then go to the panel above it and switch to the "Import" tab. It shows you all the relevant options for your file type, and once you're done you just hit "Reimport". (Depending on the size of your file and the type of processing required this can take some time.)

But part of why I'm writing this article is to give you a deeper insight into what is actually going on behind the scenes, so let's take a look at a real world example from my own game Pure Gauge, krapfen_a.png.import:

[remap]

importer="texture"
type="CompressedTexture2D"
uid="uid://b23pmtigduu0q"
path.s3tc="res://.godot/imported/krapfen_a.png-bd751845fa2b24e61f8b2183d512992b.s3tc.ctex"
metadata={
"imported_formats": ["s3tc_bptc"],
"vram_texture": true
}

[deps]

source_file="res://assets/textures/food/krapfen/krapfen_a.png"
dest_files=["res://.godot/imported/krapfen_a.png-bd751845fa2b24e61f8b2183d512992b.s3tc.ctex"]

[params]

compress/mode=2
compress/high_quality=false
compress/lossy_quality=0.7
compress/hdr_compression=1
compress/normal_map=0
compress/channel_pack=0
mipmaps/generate=true
mipmaps/limit=-1
roughness/mode=0
roughness/src_normal=""
process/fix_alpha_border=true
process/premult_alpha=false
process/normal_map_invert_y=false
process/hdr_as_srgb=false
process/hdr_clamp_exposure=false
process/size_limit=0
detect_3d/compress_to=0

There are three sections, starting with [remap]. The reason for that name will become appearent in a bit. The first two properties here indicate which importer to use, i.e. which internal class is responsible for doing the heavy lifting, and which resource type the end result will have. In this case it is set to the default, a compressed 2D texture, but your file might need to be represented as a resource of a different type, such as a Texture2DArray. Or in the case of 3D models, you can import your file either as a scene or, if you only care about the animations it contains, directly as an AnimationLibrary resource. In the editor, you change this setting by choosing the desired resource type from the "Import As" dropdown in the import panel.

The next line is the UID, a unique identifier associated with the resource. It is automatically randomly generated by Godot and you can use it to refer to a resource inside your scripts regardless of its path, reducing headaches when you later want to move it somewhere else. If you use Godot 4.4, you may have noticed that there are now also .uid files generated for each of your scripts. Scripts are also resources, but they are not "imported" in the same way and thus don't have a .import file associated with them. In order to still make it possible to refer to scripts by a path-independent universal identifier these new files are now generated, and all they contain is the UID. I recommend reading the official blog post introducing them if you want to know more.

Next up is the path to the processed file. When an asset is imported the original file is left untouched, and a new one is generated somewhere inside the .godot/imported/ directory of your project. This is the file that's actually loaded during runtime, and the one that gets shipped to your players.

Then there's a metadata dictionary with some more related properties. To be completely honest I don't know why they aren't just regular properties like the other ones and why they have to be listed in this section, I'll update this article if I figure it out.

The next section, [deps], is used by Godot to keep track of dependencies. It lists the original source file and the paths to the generated processed files. Finally, the [params] section contains all the properties and the values you set in the import panel.

Exploring an exported project

I promised I'd tell you why the first section is called "remap". To understand this, let's export our Godot project. This typically results in an executable file accompanied by a .pck file containing all the data. In the export dialog you can actually choose to export this as a.zip file instead, and I invite you to try that since it allows you to easily look inside and see what an exported Godot project actually looks like.

What you'll find is a folder structure that mirrors your project directory, but something is different. For one, the original files are gone and only the .import ones remain! This is because of what I told you in the beginning: the intermediate files in formats such as PNG aren't needed to run the game and are stripped upon export, leaving only the processed versions inside the .godot/imported/ folder (on some platforms it may be hidden, in that case configure your file manager to show hidden directories). But let's say you have load("res://assets/textures/items/food/krapfen_a.png") in one of your scripts - the game still needs to find an imported asset when referring to it via its resource path, which is why the folder structure and import files still exist. Godot looks inside the data pack and searches for the given path (suffixed by .import). The [remap] section contains all the information required to find the actual file and load it with the correct importer - it remaps the original project structure to the internal one.

When you open one of these .import files from the exported project you'll notice that the other sections are gone - [deps] and [params] are only required when working on the project and are stripped from the shipped game.

I also told you earlier that scripts are special kinds of resources. They are not "imported", meaning you won't find them in the .godot/imported folder. Which is not to say they aren't processed - GDScripts are compiled to binary formats with the .gdc extension. They're also accompanied by a file ending in .remap, which just contains a line pointing to the .gdc next to it. This may seem redundant, but I assume it is that way to support all sorts of different scripting languages other than GDScript.

[remap]

path="res://scripts/items/item.gdc"

I hope you find this adventure as exciting as I do, because we're continuing our exploration of the exported project! None of this is strictly necessary knowledge, but I think there's great value in understanding what's going on under the hood, and it will come in handy later when we'll tackle customizing the import process. It's also crucial to understand if you're interested in modding since mods are applied to exported projects.

Shader files are also special in the sense that they're not "imported". Unlike GDScript, .gdshaders aren't compiled - they're simply left untouched. They should also have accompanying .uid files - they didn't in my project because I had created them before installing Godot 4.4. To fix that I had to run Project -> Tools -> Upgrade UIDs... in the editor.

.tscn and .tres files don't need to be imported either, they're already in a format Godot natively understands, and they contain their respective UIDs in the first line of the file, so they do not need associated .uid files either.

Finally, let's take another peek inside the .godot folder contained in the data pack. It is also slimmed down compared to its equivalent in the original project folder, containing only the data that's necessary during runtime. Besides the imported directory, there is also global_script_class_cache.cfg and uid_cache.bin. The former is a human readable list of dictionaries mapping global GDScript class names (the ones declared by class_name) to the path of the respective script and some additional meta data, while the latter appears to be a binary database mapping UIDs to the paths of the files they correspond to.

Conclusions

So now you know how assets are imported, where those processed files go, and what your project looks like internally when it is exported. The last point is especially critical since the main reason exported games may behave differently than when you launch them from the editor is the file structure. For instance, sometimes it makes sense to iterate over the contents of a resource folder at runtime with DirAccess.get_files_at. In Pure Gauge, I do this to get a list of every script inside a certain folder and register a single instance of each with my in-game developer console, as mentioned in my previous post. This worked fine in the editor but led to crashes when I had exported the game. The reason is that I was expecting every file in that directory to end with the .gd extension, but in the exported version those were replaced by .remap files. I hope this knowledge saves you from similar troubles in the future, and remember to test your projects early also in exported forms and not just from the editor! (Update: I have been told that you can use the ResourceLoader.list_directory method to get the contents of a path as they are in the original project, even in exported builds!)

Also, I see a lot of confusion about which files are safe to delete, which are important, and which should be committed to version control. Now that we understand the theory we can draw some conclusions:

• .import files should be checked into version control. They contain crucial data describing how the associated asset should be interpreted and used as - it can make the difference between importing something as a simple 2D texture or a cubemap used for a skybox. If you delete a .import file or move the original asset and forget to bring the .import file along, then a new one with default parameters will be generated. Which is fine if you didn't change any of the settings, but if you did then your asset might break. And if you do not commit it to VCS and someone else clones your repository then they'll have trouble running your project.

• The .godot folder is, ideally, redundant. It contains cached data used by the editor and imported resources, all of which should be derivable from your regular project files. If you delete it, Godot will simply regenerate the contents next time you open the project. Therefore you should not commit it to version control. As long as the .import files remain the import process can be repeated correctly. (Note that I said ideally because in practice Godot can bug out and make the cached data inconsistent, causing lots of problems. That's why when you encounter strange issues, one of my first recommendations is to completely delete the .godot folder and let it be regenerated. I've encountered cases where that didn't help and the cache was generated incorrectly again. In that case, my go-to strategy is crying.)

• .uid files should be committed to VCS. If deleted, a new UID will be randomly generated, breaking any references that used the old one. That said, if you never rely on UIDs then I think it doesn't matter much, but better be safe than sorry.

• There's one more secret folder I kept from you - on Linux, it's ~/.cache/godot. It also contains cached data and should be safe to delete. In fact, in the cases where deleting .godot didn't do anything this sometimes saved me. It's very mysterious and I don't know why it exists, but it contains project-specific data that can also lead to inconcistencies when something bugs out (although I haven't encountered such issues in a long time, looks like they've mostly been fixed). According to one of the developers this may be a holdover from earlier version where the .godot folder hadn't existed yet, and it could be consolidated in the future.

And that covers the first part of this series! The importers built into the engine and supported file formats are fine and all, but what if we invented a new format that needs to be post-processed? Or what if the way the importer generates Godot scenes from 3D models doesn't fit our needs and we want to customize the output? And what's that "Advanced" button in the import panel for 3D assets? That will be the topic of a future post! In the meantime, don't forget to also check out the official documentation.

Thank you very much for reading this far! I hope I was able to clarify some of the question that have been keeping you up at night. As always, if you have further questions, feedback, or suggestions, you can reach me on Mastodon, Bluesky, or via email at rie at proton dot me. And if you don't want to miss the next entry in the series, you can subscribe to the Atom feed.