Jump to content

Wiki Task #008: Central game data source

Recommended Posts

This forum thread documents the activities related Wiki Task #008 as listed on the Community portal.


Recently, Sneaky and NoOriginality got into a discussion over at Discord #wiki. Two ideas came out of the discussion:

  • Instead of maintaining numerical game values on the wiki pages manually (e.g. weapon statistics or ticket/score counts), we should store that data centrally in one file/place and use that central data to display it on the wiki pages with some automatic mechanism. This way we only have to maintain one source of data and all pages will automatically (magically!) be updated. This is particularly useful for translations, as values no longer would have to be updated separately on all translated pages. This idea has become Wiki Task #008 (see full list of all wiki tasks on the Community Portal).
  • An additional idea came up: We should automatically pull as much of this game data/stats from Squad game files with some tool (aka "data mining") instead of manually deducing it in-game. This way we can quickly and accurately update the statistics and data on the wiki every time a new patch comes out. Of course, this idea goes well with the first idea and would be a natural second step. To deal with these two ideas one at a time, this second idea has become Wiki Task #009 - read more about it in this thread.

Both ideas are not entirely new; we had internally discussed them with the devs in the past (they liked the idea!) and we considered it a future possiblity. Maybe now is the right time to start this little project.


Organisating work on the wiki:
As wiki editors we have several tools at our disposal to coordinate and communicate our work on the wiki. These tools are particularly necessary/useful for tasks that are complex, involve multiple people, affect many wiki pages or the fundamental structure of the wiki, or take considerable time to complete.

  1. The Community Portal is the number one hub for all editorial things. Start here. It has a list of tasks and to-dos and shows all communication channels we have available. It's a bit out of date, but we can use it more if wiki editors will use and follow it too.
  2. For complex tasks, it's useful to use the forum for documentation and coordination (this is why I created this thread here). We can describe what we aim to do and we can document and share important progress and insights here. We can also have complex discussions here which would get a total mess on Discord. The orginal poster just needs to moderate the discussions and keep the first post up-to-date with all relevant information. For complex tasks I also strongly recommend to write up the aim and action plan (and maybe an open point list). It helps the person doing the task to think everything through before he starts. If a team of people works on the task, having an aim and plan is a necessity to coordinate the work of each member. Also, all of this communicates the activities to all wiki editors and the community. Who knows, maybe someone will offer their expert help for a specific step.
    1. Aim: Describe what you want to achieve. Give a good and thorough description.
    2. Action plan: Think about all the things that need to be done, then bring it into a logical and chronological order. Make notes of what is important to remember when you are doing these steps. When you write the first draft of the action plan, it's ok if it's not 100% perfect and has missing steps or lacks details. You can add to the plan as you go along.
  3. In the early Wiki days, we started this Google document to list all the wiki pages and to coordinate and track progress. Initially, we used this document to complete the transfer of the text from the Squad User Manual to the wiki, but we can use this spreadsheet for other tasks and management, too. It's very useful to keep track and have an overview of what needs doing and what has been completed, page by page. We can add as many new columns to this spreadsheet as we need. We can also create additional Google documents if we need them (please let me add them, don't just share your own Google documents). Anyone can view the document. I will only give write permission to wiki editors assigned to specific tasks.
  4. Discord should be the main tool for direct communication and close cooordination. If we really do this task, we will probably kick it off with a voice meeting on Discord to discuss everything with the assigned wiki editors (as I did for Wiki Task #006 - sadly, this task was never completed).
  5. Recent changes is an excellet way to check what has happened on the wiki recently. Alternatively, you can also check the change history of an individual page (example: Community portal history). There is also a comparison function that lets you see the differences between the current page and older revisions - very useful! This shows you why it's so important and useful to always enter a "summary" when you do an edit of a page.
  6. We should use the wiki itself as much as possible for any of these tasks, too. MediaWiki provides useful tools. For example, you can mark a page as "stub" (and other categories) and it will be automatically be listed as a page that needs work done (see Wiki Maintenance and Stubs). This way we can also mark pages for editors to work on. We can add a description of what needs doing in the page itsef or the discussion page of the page.

We can create additional "management and collaboration tools" if we need them. But for now, lets try to use the tools described above.


Volunteers: But most importantly, for task this big, it needs someone (or multiple people) to take up the assignment and start working on it. I don't have the time to do neither of these two tasks, so unless someone from the community picks it up, it will not get done.

Aim for Wiki Task #008:

  • Create/add automatic mechanism to the wiki to achieve the following:
    • Manage all numerical game values centrally in one place or file on the wiki ("the source").
    • Wiki pages pull data from the source in order to display it on a page.
    • When we update the data in the source, all pages will automatically get updated. We no longer have to edit the pages individually to update the data on each page.
  • We should try to use existing mechanism/scrips/wiki modules/wiki add-ons - no need to reinvent the wheel.
  • We should keep in mind Wiki Task #009 as we want to integrate both together.
  • Our solution has to work with our Gamepedia MediaWiki. We cannot install whatever modules or add-ons or scripts we want, we need to coordinate with Gamepedia staff what is available or what could be added.

Open points:

  1. Who will do this task? Volunteers?
  2. Do we want to do this for numerical values only? We should list specific data examples and pages to see where we would apply this mechanism and what types of data are we talking about.

Action Plan:

Just as a suggestion and starting point for discussion, let me outline how we could approach this task.

  • 1) Phase 1: Preparation
    • 1.1) Research: How could it be done? How have other MediaWikis done it? Lets compile a list of what we can find on this. I can also talk to the Gamepedia staff to seek their advice; also, I am direct communication with all the other wiki admins at Gamepedia - I can also poke them.
    • 1.2) Evaluation: Which of our wiki pages would we use this mechanism on (lets list them all up)? And what types of data would we use (lets list them all up)?
    • 1.3) Design: Decide on how we implement it. Maybe it makes sense to use templates.
  • 2) Phase 2: Implementation.
    • 2.1) Do what is necessary to implement it. Probably test it first with a few test pages, then roll out for all relevant wiki pages (English pages first, translations later).
    • 2.2) Maybe it will be necessary to standarising some of the wiki pages. We should probably making more use of templates, too. (See also Wiki Task #006: Weapon pages)
  • 3) Phase 3: Wrap up
    • 3.1) Documenting how wiki editors need to use this mechanism. Document the mechanism itself too, in case it needs maintenance.
    • 3.2) Discuss with the devs a process by which they could provide us the latest data after a new patch has come out so we can quickly update the source.
    • 3.3) Proceed with Wiki Task #009.

Assigned wiki editors:
(will be added once we have found volunteers)



  • 2017-07-15: Idea was documented in this thread.
  • 2017-06-24: Idea was proposed by Sneaky on Discord.
Edited by Usgu

Share this post

Link to post
Share on other sites

I did some research into existing scripting tools and particularly those that are already supported by Gamepedia. The following are my findings; not a complete list, not an in-depth research. There are many other Gamepedia wikis that use some of the following technologies. There is great collaboration and info sharing between all the wiki admins, so it's possible to get in touch with them to find out more or to get help for getting started.



There are various scripts available to automate handling of data for MediaWikis.


1. Semantic MediaWiki (SMW)


Due to SMW's incompatibility issues with many other extensions and systems, Gamepedia no longer supports SMW.


2. Cargo



Cargo basically are infobox-style templates that both define a data structure and then store their data in a Cargo database tables so it can be easily queried.


3. Scribunto




Scribunto allows the use of the scripting language Lua (or other languages) in a MediaWiki.


4. Page Forms



Parsed forms that create and edit pages that use infobox-style templates.



It's also possible to run automated bots that perform tasks. There are multiple bots available.


1. PyWikiBot (PWB)


2. AutoWikiBrowser(AWB)


Other stuff

I also came across the following, but not sure what it does and if it's useful to us.


1. DynamicPageList3 (DPL)

Edited by Usgu

Share this post

Link to post
Share on other sites

I'd like to give specific examples of what we are trying to achieve. Below are types of data we want to export from Squad's SDK and then import to the wiki and use it on the following pages/tables/data. Looking at these complex data types and our many tables and infoboxes, I am afraid this will become fairly complicated, so choosing the right tool to do the job is important. I am not sure if Cargo would be up for the job or we need something more powerful like Scribunto.


Map data and images

  • Map hi-res images with grids plus showing each layout (with flags and flag names; maybe also with vehicle spawns for BOTH teams) for each game mode, list factions per layout, list of assets per layout, list of flag names per layout, starting tickets for each team per layout).

Example: https://squad.gamepedia.com/Gorodok
Plenty more: https://squad.gamepedia.com/Maps
See also https://squad.gamepedia.com/WP_-_Helpful_info_for_server_admins#Idea_1:_maps_overview_for_map_rotation


Vehicle stats

  • List of all vehicles, ideally which faction are using them. Example: https://squad.gamepedia.com/Vehicles
  • Respawn timer, number of passengers, ticket costs, supply capacity, in the future: crewkit required "yes/no").

Example: https://squad.gamepedia.com/Category:Vehicle_infosheets
Example: https://squad.gamepedia.com/Tickets#Assets_ticket_costs


Weapon stats


Deployable stats


Kit stats


Game modes/general stats

Edited by Usgu

Share this post

Link to post
Share on other sites

We now have both Cargo and Scribunto enabled for the Squad Wiki. I have started experimenting with both of them. Check out our Cargo and Scribunto learning pages.


I'll post another status update once I have more progress. We'll try to apply Cargo and Scribunto to the Maps pages first and see how this goes.

Share this post

Link to post
Share on other sites

Great idea indeed !! Playing SL means being aware of the last changes in the game mechanics (tickets loss system, respawn timers and so on) and having those numbers centralized in the same place would be GREAT !

I absolutely don't know how to manage the wiki (and don't have time for it anyway :( ) thus I only can support you Usgu and others contributors to make this "task #008" become something working and fonctional.

Share this post

Link to post
Share on other sites

I have posted the following general steps on Discord before. These steps roughly outline how to implement Cargo to a new section of the wiki. Theoretically - we haven't actually done it ever before, so this is a first. I might also have missed a step or two, or got some order wrong, but I hope this gives a general idea of what needs doing. I'll repost them here for better and easier referencing.

  1. Decide on the data elements (and their data types) you want to store and present (this might include properties that are not necessarily displayed but also important)
  2. List all pages that use this data
  3. Decide which pages will "store" the data (usually those that have infoboxes and describe the item in question) and which pages will "display" the data (usually those with overview/summary tables of that data)
  4. Decide on the specific layout/presentation of the display tables that shows the data (there might be different scenarioes/presentations/layouts for the different target pages/purposes)
  5. Create the templates for the infoboxes; these infoboxes are used for data entry; these templates use Cargo scripts for storing the data
  6. Create one Lua script to display the Cargo data; using parameter this Lua script can be made to adjust the layout/presentation of its output - this way, we only need one Lua script instead of multiples
  7. Integrate all above in the existing wiki pages
  8. Enter all the data using the infoboxes on corresponding wiki pages (if we modifiy existing infobox templates, we might already have the data on the wiki pages)

Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now