A mod is a set of code and assets that changes or augments the way a game is run. Examples of mods are:
- Music and sound packs that replace the original game music and sound.
- Graphics packs that replace the original game graphics.
- Level packs that change or replace the original game levels.
- Achievements that are awarded to the players when they complete tasks in the game.
- ROM patches that otherwise change the way the game is presented or played.
In the Retromods context, a mod can also control the emulation in a level above of the game or system that emulates it. Examples of this kind of mod are:
- Challenges that change the sequence of the game, i.e. present all Super Mario World bosses in sequence and ask the player to beat them below the given allocated time.
- Meta-achievements across games and systems, i.e. an achievement awarded when the player beats all Metroid games in all platforms.
A mod is a ZIP file
ZIP files are not streamable, but we don't need to unpack them in advance; they will be used in their compressed form by employing an API that abstracts access to their content.
The mod identifier is an URL
A mod is identified by its URL, and it can reference other mods by it. This will avoid name clashes when different people are authoring mods. The URL must use the http
scheme.
The HTTP server at the given host and port of the URL must accept GET
and HEAD
requests to the local path part of the URI. GET
requests must return the mod's ZIP package in the HTTP response, while HEAD
requests can be used to check for mod updates. Therefore, HTTP servers should return valid Last-Modified header fields.
The frontend is only required to understand the http
and file
URI schemes. The later can be used during mod development to identify mods in the user's local file system. When development finishes, the mod must be uploaded to a public repository in an unspecified way and receive a publicly accessible URL by that repository, which will be used as its identifier.
Mods can be stored in the file system, as blobs in a database, or in any other way, but they must be accessible by their URL.
Games are identified by UUID
A game is uniquely identified by its UUID. Mods can ask the frontend to load and run a specific game by its UUID.
Libretro cores are identified by name
A libretro core is uniquely identified by its name, which is the name of its binary file minus the extension. Mods can ask the frontend to load and run a game using a specific core by using its name.
Although libretro cores are a central piece of Retromods, nothing prevents a stand-alone emulator from implementing this specification. It will, however, be limited to the mods that work with the underlying emulator technology.
Each system that can be emulated also has an unique name to identify them. These names are shorts of the complete system name, i.e. snes
for Super Nintendo Entertainment System. The complete list is TBD.
Lua is easy to embed and extend, and it's fast and lean.
The ZIP file must contain a metadata.json
file with the following fields: uri
, name
, version
, description
, authors
, category
, keywords
, and dependencies
:
{
"uri": "http://retromods.org/mods/smwhd",
"name": "High-definition Super Mario World",
"version": "1.0.0",
"description": "Super Mario World remake with the best music and graphics replacements.",
"author": "Winston Churchill",
"category": "music graphics",
"keywords": "music pack replacement super mario world mozart graphics pack sprite replacement super mario world pablo picasso",
"dependencies": [
"http://retroparadise.net/super_mario_world_hd.zip",
"http://emuaudio.com/packs/supermarioaudio"
],
"supported-games": [
"03539380-3b2f-11e8-94cd-eb2885799cc5"
],
"supported-cores": [
"snes9x_libretro"
],
"supported-systems": [
"snes"
]
}
The field supported-games
, if present, contains a list of game UUIDs with which the mod can be used. If this field is missing, the mod is applicable to any game of the supported systems. The same thing is true with the supported-cores
and supported-systems
fields.
Besides metadata.json
, the ZIP file must also contain a main.lua
file with has the mod's functionality. Both metadata.json
and main.lua
must exist in the root folder of the mod ZIP.
When a mod is added to the frontend, the frontend must download it's dependencies if they aren't already available. If one of the dependencies of the mod fails to be added to the frontend for whatever reason (i.e. failed download, corrupted package), the mod must not be added.
Other than that, the only guarantees are those made by code.
All modding functionality is done via Lua code. main.lua
must return a function that will be called when a mod is activated by the frontend. This function will receive all its arguments as fields in a table passed as the first parameter:
return function(args)
-- args.vfs: the read-only virtual file system of the mod.
-- args.persistent: a read-write virtual file system for data the mod wants to
-- make persistent. This vfs should be used for user-related content, such as
-- high scores.
-- args.cache: a read-write virtual file system for persistent, but not
-- user-related, content that the mod wants to have around. Unlike the persistent
-- vfs, cache is never synchronized, and the mod must be able to recreate its data
-- if needed.
-- args.scratch: a read-write virtual file system for a temporary area that the
-- mod can use at will, but which will be erased when the mod is unloaded.
-- args.submods: a map from each of the sub mods' URL to their respective
-- read-only virtual file system.
end
A reference to the args
table or any of its fields can be saved by the mod to be used at any moment. It can also be changed in any way the mod sees fit.
When executed, this function must return a table with the events that it's interested in:
onUnload
: runs when the mod has been deactivated by the frontend.onCoreLoaded
runs when a libretro core has been loaded by the frontend.onCoreUnloaded
runs when a libretro core has been unloaded by the frontend.onContentLoad
: runs when a game has been loaded, but emulation has not been started.onContentUnload
: runs when a game has been unloaded.onEmulatedFrame
runs once per frame, right after the emulated frame has ended.onEmulationPause
: runs when the user pauses the emulation.
Note: This list is not final and can be changed as the project matures.
Each field must be a function that will be executed when the event is fired. The events are only fired to the main mod, it's up to the main mod to propagate events to its submods.
Example main.lua
:
local frontend = require 'frontend'
local json = require 'json'
return function(self)
local source = assert(self.vfs:read('/metadata.json'):result())
self.metadata = assert(json.read(source))
for url, vfs in pairs(self.submods) do
local main = assert(vfs:read('/main.lua'):result())
-- this won't break the iterator
self.submods[url] = {
vfs = vfs,
handlers = assert(frontend.run(main))
}
end
function self:onFrame()
for url, mod in pairs(self.submods) do
mod.handlers:onFrame()
end
end
return self
end
Modd receive a persistent virtual file system in the persistent
field of the argument to their main function. This file system's content is persisted between runs of the mod, and can be used to store content that cannot be lost such as high scores.
Although not required, frontends should use 3rd-party online services to keep this user data available and synchronized online. Which online service and how the frontends keep the userdata synchronized is not specified here.
Backends wanting to offer mods discovery functionality for frontends must implement a set of REST services.
Note: These services are not final and can be changed as the project matures.
Mods can be queried in many different ways:
- By game:
/mods/?games=${game-uuid-list}
. If the list has more than one game UUID, they must be separated by commas. - By category:
/mods/?categories=${category-name-list}
. - By keywords:
/mods/?keywords=${keyword-list}
. - By list:
/mods/?list=${list-name}
.list-name
can be one of:most-downloaded
: mods ordered from the most downloaded to the least downloaded.most-starred
: mods ordered from the most starred by the site users to the least starred.editor-picks
: mods ordered from the most starred by the site staff to the least starred.curated
: mods ordered from the most starred by a specific user to the least starred. This list requires&user={user-id}
to be appended to the URL, whereuser-id
uniquely identifies an user on the backend.
Backends are not required to implement all queries. Queries not implemented should return a 501 Not Implemented HTTP status code.
The result is paginated in pages of 25 mods each. While frontends are expected to check for this limit, they are not required to handle more than 25 elements in each page. In other words, frontends need not prepare to parse responses with infinite items.
To request a specific page add &page=${page-number}
to the URL. Page numbers start at 1. Any list can be reversed by adding &reversed=true
to the URI. Games, categories, keywords, and one list can be freely intermixed to provide advanced ways to filter the collection of mods.
Example of response:
{
"total-results": 150,
"page-number": 1,
"first-element-in-page": 1,
"last-element-in-page": 25,
"results": [
{
"url": "http://emuaudio.com/packs/supermarioaudio",
"name": "Music pack for Super Mario World",
"version": "1.2.1",
"description": "Complete music replacement with original game music performed live by Mozart.",
"author": "Mozart",
"category": "music",
"keywords": "music pack replacement super mario world mozart",
"dependencies": [],
},
{
"url": "http://retroparadise.net/super_mario_world_hd.zip",
"name": "Graphics pack for Super Mario World",
"version": "3.4.0",
"description": "Complete graphics replacement with pixel art made by Picasso.",
"author": "Pablo Picasso",
"category": "graphics",
"keywords": "graphics pack sprite replacement super mario world pablo picasso",
"dependencies": []
},
{
"url": "http://retromods.org/mods/smwhd",
"name": "High-definition Super Mario World",
"version": "1.0.0",
"description": "Super Mario World remake with the best music and graphics replacements.",
"author": "Winston Churchill",
"category": "music graphics",
"keywords": "music pack replacement super mario world mozart graphics pack sprite replacement super mario world pablo picasso",
"dependencies": [
"http://retroparadise.net/super_mario_world_hd.zip",
"http://emuaudio.com/packs/supermarioaudio"
]
},
Other results follow...
]
}
Additional fields are available in each result depending on the specific query:
- If
keywords
is used, each result will have akeywords
field containing a number from 0 to 1, which is the relevance of the result in relation to the other results in the same result list. A result with a value ofX
is more relevant than a result with a value ofY
ifX > Y
. - If a list if used, each result will have a
list
field containing a number, which range and meaning depends on the list:most-downloaded
: the total number of downloads since genesis.most-starred
,editor-picks
, andcurated
: a number from 0 to 1, where 0 means "no starts" and 1 means "all stars."
Games are identified with their UUIDs. To return the details of a game, the /games/${game-uuid}
can be used.
Queries that retrieve game lists are:
- By hash:
/games/hash=${hash-list}
. The hash is calculated by a system-specific algorithm that runs over the game content. - By system:
/games/?system=${system-name-list}
. - By keywords:
/games/?keywords=${keywords}
.
Each game has at least an UUID, its name, and the system name:
{
"uuid": "03539380-3b2f-11e8-94cd-eb2885799cc5",
"name": "Super Mario World",
"system": "snes"
}
The resulting lists must be formated in the same way as the lists of mods shown above. As with mod queries, each result may have additional fields depending on how the query was made.
The backend is free to return additional information about the games if present in their databases, i.e.
- A description of the game.
- The year it has been released.
- The name of the recommended libretro core to play the game.
- The UUIDs of games in the same series.
- URLs that can be used to view or download game art, screenshots, and videos.
Some of those information may be specified in the future to have a specific key in the resulting JSON.
Frontends should not offer the download of copyrighted games, nor should they offer directions to places where they can be downloaded.
The backend must provide services to query libretro cores to be used with games and mods.
To retrieve the list of libretro cores that support a given system, use the /cores/?system=${system-name-list}&platform={platform}
URL.
To retrieve information for a specific version of a core, add &version=${version}
to the URL, where version
is in the core-specific version format. If version
is not informed, the result list only includes the latest versions.
As an example, the resulting list of the query for the /cores/?system=atari2600&platform=linux%2Fx86_64
could be:
[
{
"name": "Stella",
"downloads": [
{
"version": "3.9.3",
"platform": "linux/x86_64",
"url": "http://buildbot.libretro.com/nightly/linux/x86_64/latest/stella_libretro.so.zip"
}
]
}
]
Note: System and platform names are TBD.
Lua scripts will have bindings available to communicate with the emulated system, its components, the frontend, and the backend. The exact set of functionalities is TBD.
Lua is extended with some types that represent things that cannot be represented natively or in a efficient way in the language.
bytes
represent a block of bytes. It has the following methods:
get8(address)
: returns the byte value ataddress
. Byte values go from 0 to 255.get16le(address)
: returns the word value ataddress
, in little-endian format. This is equivalent toget8(address + 1) << 8 | get8(address)
.get32le(address)
: returns the double word value ataddress
, in little-endian format. This is equivalent toget16le(address + 2) << 16 | get16le(address)
.get16be(address)
: returns the word value ataddress
, in big-endian format. This is equivalent toget8(address) << 8 | get8(address + 1)
.get32be(address)
: returns the double word value ataddress
, in big-endian format. This is equivalent toget16be(address) << 16 | get16be(address + 2)
.get8bcd(address)
: returns the byte value ataddress
, in BCD format. This is equivalent to(get8(address) >> 4) * 10 + (get8(address) & 15)
. No checks are made to see if the value is a valid BCD value.get16bcdle(address)
: returns the value of the word value ataddress
, in BCD little-endian format. This is equivalent toget8bcd(address + 1) * 100 + get8bcd(address)
.get32bcdle(address)
: returns the value of the double word value ataddress
, in BCD little-endian format. This is equivalent toget16bcdle(address + 2) * 10000 + get16bcdle(address)
.get16bcdbe(address)
: returns the value of the word value ataddress
, in BCD big-endian format. This is equivalent toget8bcd(address) * 100 + get8bcd(address + 1)
.get32bcdle(address)
: returns the value of the double word value ataddress
, in BCD big-endian format. This is equivalent toget16bcdle(address) * 10000 + get16bcdle(address + 2)
.getFloatle(address)
: return the value of the 4-byte float ataddress
, stored in little-endian.getFloatbe(address)
: return the value of the 4-byte float ataddress
, stored in big-endian.size()
: the total number of bytes in the object.clone()
: returns a clone of the object. The contents are copied, so the cloned instance is completely independent of the original object.view(offset, size)
: creates a newbyte
with a view of the object.offset
andsize
default to 0 andsize()
, and allow copies of pieces of the original object. Content is not copied, the view points to the same content as the original object. In other words, changes in the content of the original object are observable in its views.concat(blocks...)
: this static method creates a newbyte
that is the concatenation of the givenbyte
instances. Likewise the views, content is not copied, accesses go to the correctbyte
.create(size, value, rw)
: this static method creates a newbyte
which hassize
bytes all set tovalue
. Ifrw
istrue
, writes to the block are persisted, otherwise the block is optmized but does not persist writes.
byte
instance also have setX(address, value)
methods with the same variations of the getX(address)
ones.
Lua scripts are not allowed to access the local file system, for security reasons. However, it's necessary that they sometimes reference objects that exist in the local file system, like the file containing the response body of a the result of a HTTP GET
. In these cases, the local path to the file system is represented by localpath
, which is an opaque object that can only be decoded by a small, secure set of native functions.
localpath
instances are never created from Lua, and don't have any methods. They can only be returned from and passed to native functions.
local frontend = require 'frontend'
run(sourceCode)
: runs the script and returns its return value to the caller.saveState()
: returns a snapshot of the game state as abyte
, which can be used later to restart the game from the point the snapshot was taken.loadState(state)
: restart the game at the point the snapshot was taken. Snapshots are game and core dependent, loading a snapshot from a game and/or system into another game and/or system may cause the frontend to crash.patch(localPath, patchData)
: patches a content atlocalPath
using the IPS, BPS, UPS, or PPF patch atpath
. Returnstrue
on success, ornil
plus an error message on error. Patches are not permanent; all modifications to a content are lost when the content is unloaded.
local async = require 'async'
Asynchronous work will always return a waitable object when scheduled. This object can be used to wait for the completion of the associated work, and contains information about its completion.
To wait for the completion of work, use the wait()
method of the associated waitable. wait()
is a blocking call, and will put the script to sleep until the waitable finishes. wait()
returns two results, the result of the asynchronous operation if it succeeds, or nil
plus a string describing the error. It's legal to use assert(waitable:wait())
to synchronize on a waitable object, while checking for its success and returning its result at the same time.
To test whether a waitable has finished or not without blocking, use finished()
method, which returns true
if it has, and false
otherwise. Waitable objects also have a result()
method which is identical to wait()
, use one or the other according to the semantics of your code.
Limits on the number of concurrent asynchronous work being performed may be limited depending on the system. Do not assume the system will always perform all asychronous work in parallel.
waitAll(set, onFinished)
: waits for the completion of all waitable objects in the list, successful or not. For each waitable that is completed, theonFinished
function will be called, if present, with the waitable object, and its value in the set.waitAny(set)
: wait for the completion of any one of the waitable objects in the list, and return the one that was completed along with its value in the set.
On both functions the waitable object is removed from the set.
The mod can sleep for some time via the pause
function:
pause(ms)
: returns a waitable that will only finish afterms
milliseconds have passed.
local http = require 'http'
get(url, dicardResponse)
: starts downloading asynchronously via HTTP and returns an waitable object. When the download completes, the waitable object will have astatus
field in theresult
property with the HTTP response code, and alocalPath
field which contains the local path to the contents of the download if it was successful. IfdiscardResponse
istrue
, then the HTTP response will not be saved, andlocalPath
will benil
.
local filesystem = require 'filesystem'
File system objects cannot be created from Lua. They are returned by some native methods to let Lua scripts interact with some files and folders in the file system.
filesystem
objects have the following methods.
read(path, offset, count)
: starts reading the contents of the entrypath
in the file system. The result of the waitable object if of thebytes
type.offset
defaults to 0, and be used to specify the position of the file to start reading.count
defaults to makefileRead
read all the entry content beginning atoffset
.write(path, contents)
: starts writingcontents
, which can be abytes
user data or a string, into the file atpath
.
local hash = require 'hash'
Hashes can be computed by creating a hash
object, and feeding strings or byte
objects to its update
method. The computed hash is returned by the final
method as a byte
instance. To return the hash as a hexadecimal number in a string, use the tostring
method. Note that both final
and tostring
finishes the hash
object, so it cannot be further used.
create(type)
: this static method creates a newhash
object of the giventype
. The type must be a string with the name of a supported hash.update(content)
: updates the hash object with a string orbytes
object.final()
: returns the hash as abyte
object.tostring()
: returns the hash as a hexadecimal number in a string.
The following hashes are supported: MD5, SHA-1, SHA-256, SHA-384, SHA-512, CRC-32, CRC-64, and Adler-32.
local json = require 'json'
unserialize(json)
: translates the JSON in the given string to a Lua table. Returnsnil
plus an error message in case of failure.serialize(data)
: translates the Lua table indata
to a string containing the equivalent JSON and returns it. Returnsnil
plus an error message in case of error, i.e. circular references.
Since Lua can't differentiate between a key with the nil
value in a table, and a key which doens't exist in a table, json.null
is used as the value of keys which are null
in the JSON input. It should also be used to set keys in tables that should be null
when serializing back to JSON.
local timer = require 'timer'
Mods can create timers to control the passing of time. Their precision is milliseconds, and they can count upwards or downwards.
create()
: creates a timer that counts upwards.create(ms)
: creates a timer with the time set toms
milliseconds, that counts downwards.
Timers have these methods:
start()
: starts the counting.stop()
: stops the counting. Counting can be started again with a call tostart
.millis()
: returns the current millisecond.second()
: return the current second.minute()
: return the current minute.hour()
: return the current hour.onTime(ms, func)
: executesfunc
, passing the timer as its only argument, when the time reachesms
milliseconds. Note that this is not the amount of milliseconds passed, but an actual point in time inside the timer's range, i.e.onTime(0, func)
will trigger when timer that counts down reaches 0. If the point in time has already passed,onTime
will not trigger. Each call toonTime
overrides the last one.
I read the whole thing, I think it's all good :)
I don't have too many questions, actually I think it all sounds great because it opens up a lot of possibilities.
But, what do we do now?