Smaller mappacks, less download

[IzNoGoD]

This is a long post about how to create very small mappacks for your server. It relies on a modified version of the extension (...) and allows clients to only download the currently playing map instead of every map you have on the server.

It is currently running on the JumpersHeaven server (/connect jh.killtube.org) without issues and as such I decided to release everything.

Prerequisits:
- A properly functioning brain capable of more advanced computer stuff.
- Python 3.3
- 7zip
- Proper knowledge of codscript
- Knowledge of libcod and compiling it
- A few hours of your time
- fs_game for your mod
- Probably more

First step is to install python 3.x if you dont already have it installed.

Create a folder on C:/, I used JH2 (C:/JH2 is actually hardcoded in the python script, if you want to use some other folder you have to change the script)

Create a folder "stock" in said folder, copy all (including localized_english) iwds from your cod2 directory to there and extract them in such a way the C:/JH2/stock/maps folder exists (extract on the spot, not to a subdir)

Create a folder called "merged" in C:/JH2/ folder, extract all your mappacks to said folder

Edit all your soundaliases files so every single map has a mapname.csv soundalias (there is NO check for this, you have to do it by hand)
Edit all your soundaliases so they have a proper loadspec for every sound file (is technically not entirely needed, but the script is instructed to throw warnings if improper loadspecs are used which makes the output quite unreadable)

Create a folder "add_models" and a folder "add_shaders" in C:/JH2/

Look through the scripts of your maps to see if any additional xmodels or images are used.
- If any additional xmodels are used, store their names in a mapname.txt files (do this per-map) in the add_models folder, layout (taken from jm_pier_2.txt):

xmodel/prop_lamp01_on
xmodel/prop_lamp01
xmodel/cod4_sunglasses
xmodel/pier2_ancient_coin
xmodel/caspi_desert_gold
xmodel/de_v

- Do the same for images the scripts use, store in same manner in add_shaders. Example:

pier2_credits_map_name
objective
pier2_manifest_paper
cj_banner2
unreal_jumperz_crew_rzl128x128
lockpicking_cylinder_0
lockpicking_cylinder_1
lockpicking_cylinder_2
lockpicking_cylinder_3
lockpicking_cylinder_4
lockpicking_cylinder_5
lockpicking_cylinder_6
lockpicking_cylinder_7
lockpicking_cylinder_8
lockpicking_cylinder_9
lockpicking_pin
lockpicking_background

Note there is no folder here, just the material (!) name.

Create a .bat file with the following contents in C:/JH2 as zip_packs.bat:

for /D %%d in (*.*) do 7z a -tzip "%%d.iwd" ".\%%d\*"

This will later on pack the files into their .iwd packs.

Now, make sure the create_packs2.py file is also in C:/JH2/. You can open a commandprompt there if you like, or just double-click the .py file, it should execute in a commandwindow. Note that doubleclicking does not allow you to see any errors with 7zip, so i advice you to open a commandprompt.

The script now starts to do its magic. It does the following:
- Find all .d3dbsp files in merged/maps/mp folder
- Find all corresponding .csv files in merged/maps/mp to search loadingscreens, read them and add the found material files to a material list
- Reads the d3dbsp file and finds all materials and all xmodels used within that certain map, then adds them to the material and xmodel lists respectively
- Opens up all xmodel files (if not found in the stock files) and reads them to find all materials used by the xmodel. Note: this sometimes seems to fail, and the script will throw a "bad stuff happened for materialfind xmodelname mapname" error. Afaik this is not a real issue, as it seems the xmodel in question is not using any materials.
- Reads xmodel to find xmodelsurf/xmodelparts used by it, then adding it to lists
- Opens and reads the mapname.csv soundalias file (This is why you should name your soundalias file the same as the mapname), adds all sounds to a list
- Goes through all material files and finds the corresponding .iwi files

After these steps, it goes through all lists for that map and searches the stock folder for those files. If not found, it searches the merged folder for them. If still not found, it throws an error which you should try to fix.

If a .iwi file is listed as not found, you SHOULD fix it for it MIGHT crash the server, which it surely will if the file in question is part of a loadscreen.

Now that all maps have their own folder, the packing of the maps can begin. Pressing enter instructs 7zip to pack all separate maps into separate .iwd files (JumpersHeaven server has 200ish iwd files at the moment). These will be in the packs directory, automatically created by the script (and deleted once the script restarts, so be careful)

After the script completes, you will have a couple of folders added to the C:/JH2/ folder, namely:

- Packs
- soundaliases
- empty_files
- filelist

Packs:
Contains all .iwd files of the individual maps, as well as the extracted counterparts in separate folders

soundaliases
Contains all read soundaliases files. This should be added to your mod iwd file, which should be zzz_something.iwd

empty_files:
Contains all files present in the packs, but as 0byte files, in order to fool cod2 into thinking the files are actually present. Should be packed into a 000something.iwd file as a client iwd in your fs_game

filelist:
Contains a lot of .txt files with 1 line of text each. This is the list of .iwd files needed to run a certain map and opens up doors to shared image packs for multiple maps, but it is not a requirement to change anything to this. Should be put in fs_game/scriptdata/mapfiles/ on your server and, if your server runs stock maps, accompanied by a couple of empty .txt files named mp_toujane.txt, mp_carentan etc.

Now the packing of your maps is done it's time to start with the actual modding. You'll need a modified version of the libcod extension, with this present as function 602:

int gsc_system_command()
{
	char* cmd;
	if (stackGetNumberOfParams() < 2) // function, command
	{
		printf_hide("scriptengine> ERROR: please specify atleast 2 arguments to gsc_system_command()\n");
		return stackPushUndefined();
	}
	if (!stackGetParamString(1, &cmd))
	{
		printf_hide("scriptengine> ERROR: closer(): param \"cmd\"[1] has to be a string!\n");
		return stackPushUndefined();
	}
	system(cmd);
	return stackPushInt(0);
}

This should allow cod2 to directly access linux shell commands in order to dynamically link/unlink the .iwd files into your fs_game folder

As this script totally breaks stock map functions, the /rcon map and /rcon devmap commands stop working and should be replaced by another function. As kung is currently looking into possible solutions for this I'm not gonna give you a function to replace these. If you require such a function, write it yourself.

The voting however HAS been rewritten by me using the callback playercommand supplied by libcod. Your codecallback_playercommand in _callbacksetup should look like this:

CodeCallback_PlayerCommand(args) //depends on the extension, put it in callbacksetup.gsc
{
	args = fixChatArgs(args);
	if(args[0] == "vote")
	{
		if(isdefined(args[1]) && args[1].size)
		{
			if(args[1] == "yes")
			{
				self vote(true);
				return;
			}
			else if(args[1] == "no")
			{
				self vote(false);
				return;
			}
		}
	}
	if(tolower(args[0]) == "callvote")
	{
		self callvote(args);
		return;
	}
	std\utils::ClientCommand(self getEntityNumber());
}

in which you can put the vote commands in a separate .gsc file. These functions are:

init() //call on mapstart
{
	level.izno_votetime = 60 * 1000; 
	level.izno_vote_cooldowntime = 60 * 1000;
	level.nextmap = ??; //i dunno, i use my own mysql nextmap function here. Make sure to fill this out anyway, without it everything crashes and burns.
}

This should be read carefully as the level.nextmap is NOT set yet but SHOULD be set. I'm too lazy to find which cvars to read for this and am currently using a mysql function for nextmap.
Furthermore, these functions should be included:

onvote(args)
{
	if(isdefined(args[1]) && args[1].size)
	{
		switch(tolower(args[1]))
		{
			case "map":
			case "typemap":
			{
				if((args[1] == "map" && isdefined(args[2])) || (args[1] == "typemap" && isdefined(args[3])))
				{
					if(args[1] == "map")
						num = 2;
					else
						num = 3;
					mapname = args[num];
					fid = openfile("mapfiles/" + mapname + ".txt", "read");
					if(fid != -1)
					{
						closefile(fid);
						self startmapvote(mapname);
					}
					else
						self iprintln("No matching mapnames found");
					return;
				}
			}
			case "map_restart":
			{
				self startmap_restartvote();
				return;
			}
			case "g_gametype":
			{
				self iprintlnbold("You are not allowed to change the gametype");
				return;
			}
			case "map_rotate":
			{
				if(isdefined(level.nextmap))
					self startmap_rotatevote();
				return;
			}
			case "tempbanuser":
			case "kick":
			{
				if(isdefined(args[2]) && args[2].size)
					self startkickvote(args[2]);
				return;
			}
			case "tempbanclient":
			case "clientkick":
			{
				if(isdefined(args[2]) && args[2].size)
					self startclientkickvote(args[2]);
				return;
			}
		}
	}
}

cancallvote()
{
	if(!getcvarint("g_allowvote"))
	{
		self iprintln("Voting is disabled");
		return false;
	}
	if(isdefined(level.izno_vote) && level.izno_vote["endtime"] > gettime())
	{
		self iprintln("A vote is already in progress!");
		return false;
	}
	else if(isdefined(level.izno_vote) && level.izno_vote["cooldown"] > gettime())
	{
		self iprintln("A vote was called recently");
		self iprintln("Please wait before calling another one");
		return false;
	}
	return true;
}

startmap_restartvote()
{
	if(!cancallvote())
		return;
	level.izno_vote = [];
	level.izno_vote["veto"] = false;
	level.izno_vote["type"] = "map_restart";
	level.izno_vote["endtime"] = gettime() + level.izno_votetime;
	level.izno_vote["in_favor"] = 1;
	level.izno_vote["opposed"] = 0;
	iprintlnbold("A vote has started");
	iprintln(self.name + "^7 has called a vote to restart the map");
	players = getentarray("player", "classname");
	for(i = 0; i < players.size; i++)
		players[i].izno_vote = undefined;
	self.izno_vote = true;
	level thread monitorvote();
}
startmap_rotatevote()
{
	if(!cancallvote())
		return;
	level.izno_vote = [];
	level.izno_vote["veto"] = false;
	level.izno_vote["type"] = "map_rotate";
	level.izno_vote["map"] = level.nextmap;
	level.izno_vote["endtime"] = gettime() + level.izno_votetime;
	level.izno_vote["in_favor"] = 1;
	level.izno_vote["opposed"] = 0;
	iprintlnbold("A vote has started");
	iprintln(self.name + "^7 has called a vote to start the next map");
	players = getentarray("player", "classname");
	for(i = 0; i < players.size; i++)
		players[i].izno_vote = undefined;
	self.izno_vote = true;
	level thread monitorvote();
}

startclientkickvote(num)
{
	if(num != 0 && int(num) + "" != num)
	{
		self iprintln("Invalid client number");
		return;
	}
	user = undefined;
	players = getentarray("player", "classname");
	for(i = 0; i < players.size; i++)
	{
		if(players[i] getentitynumber() == int(num))
		{
			user = players[i];
			break;
		}
	}
	if(isdefined(user) && self == user)
		self iprintln("You cannot vote to kick yourself");
	else if(isdefined(user))
		self startkickplayervote(user);
	else
		self iprintln("No users found");
}
	
startkickvote(name)
{
	if(!cancallvote())
		return;
	found = 0;
	user = undefined;
	players = getentarray("player", "classname");
	for(i = 0; i < players.size; i++)
	{
		if(players[i].name == name)
		{
			found++;
			user = players[i];
		}
	}
	if(found == 1)
	{
		if(self == user)
		{
			self iprintln("You cannot vote to kick yourself");
			return;
		}
		self startkickplayervote(user);
	}
	else if(!found)
		self iprintln("No users found by the name " + name);
	else
		self iprintln("Multiple users found by the name " + name);
}

startkickplayervote(user)
{
	level.izno_vote = [];
	level.izno_vote["veto"] = false;
	level.izno_vote["type"] = "kick";
	level.izno_vote["player"] = user;
	level.izno_vote["endtime"] = gettime() + level.izno_votetime;
	level.izno_vote["in_favor"] = 1;
	level.izno_vote["opposed"] = 1;
	iprintlnbold("A vote has started");
	iprintln(self.name + "^7 has called a vote to kick " + user.name);
	players = getentarray("player", "classname");
	for(i = 0; i < players.size; i++)
		players[i].izno_vote = undefined;
	self.izno_vote = true;
	user.izno_vote = false;
	level thread monitorvote();
}

startmapvote(map)
{
	if(!cancallvote())
		return;
	level.izno_vote = [];
	level.izno_vote["veto"] = false;
	level.izno_vote["type"] = "map";
	level.izno_vote["map"] = map;
	level.izno_vote["endtime"] = gettime() + level.izno_votetime;
	level.izno_vote["in_favor"] = 1;
	level.izno_vote["opposed"] = 0;
	iprintlnbold("A vote has started");
	iprintln(self.name + "^7 has called a vote to change the map to " + map);
	players = getentarray("player", "classname");
	for(i = 0; i < players.size; i++)
		players[i].izno_vote = undefined;
	self.izno_vote = true;
	level thread monitorvote();
}

vote(side)
{
	if(isdefined(self.izno_vote))
	{
		self iprintln("You have already voted");
		return;
	}
	else if(isdefined(level.izno_vote) && level.izno_vote["endtime"] > gettime())
	{
		if(side)
		{
			self.izno_vote = true;
			level.izno_vote["in_favor"]++;
		}
		else
		{
			self.izno_vote = false;
			level.izno_vote["opposed"]++;
		}
	}
	else
		self iprintln("No vote in progress");
}

checkvotenotended()
{
	if(level.izno_vote["veto"])
		return false;
	players = getentarray("player", "classname");
	in_favor = 0;
	opposed = 0;
	total = 0;
	for(i = 0; i < players.size; i++)
	{
		if(isdefined(players[i].izno) && players[i].izno["afk_timer_vote"] > gettime())
		{
			if(isdefined(players[i].izno_vote))
			{
				if(players[i].izno_vote)
					in_favor++;
				else
					opposed++;
			}
			total++;
		}
	}
	level.izno_vote["in_favor"] = in_favor;
	level.izno_vote["opposed"] = opposed;
	level.izno_vote["total"] = total;
	if(gettime() < level.izno_vote["endtime"] && level.izno_vote["in_favor"] <= level.izno_vote["total"] / 2 && level.izno_vote["opposed"] < level.izno_vote["total"] / 2)
		return true;
	return false;
}

monitorvote()
{
	players = getentarray("player", "classname");
	remaining = (level.izno_vote["endtime"] - gettime()) / 1000;
	level.izno_vote["cooldown"] = gettime() + level.izno_vote_cooldowntime;
	oldtext_voted = "";
	oldtext_notvoted = "";
	text_voted = "";
	text_notvoted = "";
	while(checkvotenotended())
	{
		switch(level.izno_vote["type"])
		{
			case "map":
			{
				text_voted = "^3Vote(" + int(remaining) + "): Map: " + level.izno_vote["map"] + "^3\nYES: " + level.izno_vote["in_favor"] + " NO: " + level.izno_vote["opposed"];
				text_notvoted = "^3Vote(" + int(remaining) + "): Map: " + level.izno_vote["map"] + "^3\nYES(F1): " + level.izno_vote["in_favor"] + " NO(F2): " + level.izno_vote["opposed"];
				break;
			}
			case "map_restart":
			{
				text_voted =  "^3Vote(" + int(remaining) + "): Restart map \nYES: " + level.izno_vote["in_favor"] + " NO: " + level.izno_vote["opposed"];
				text_notvoted = "^3Vote(" + int(remaining) + "): Restart map \nYES(F1): " + level.izno_vote["in_favor"] + " NO(F2): " + level.izno_vote["opposed"];
				break;
			}
			case "map_rotate":
			{
				text_voted = "^3Vote(" + int(remaining) + "): Next map: " + level.izno_vote["map"] + "^3\nYES: " + level.izno_vote["in_favor"] + " NO: " + level.izno_vote["opposed"];
				text_notvoted = "^3Vote(" + int(remaining) + "): Next map: " + level.izno_vote["map"] + "^3\nYES(F1): " + level.izno_vote["in_favor"] + " NO(F2): " + level.izno_vote["opposed"];
				break;
			}
			case "kick":
			{
				if(isdefined(level.izno_vote["player"]))
				{
					text_voted = "^3Vote(" + int(remaining) + "): Kick: " + level.izno_vote["player"].name + "^3\nYES: " + level.izno_vote["in_favor"] + " NO: " + level.izno_vote["opposed"];
					text_notvoted = "^3Vote(" + int(remaining) + "): Kick: " + level.izno_vote["player"].name + "^3\nYES(F1): " + level.izno_vote["in_favor"] + " NO(F2): " + level.izno_vote["opposed"];
					break;
				}
				else
					break;
			}
		}
		if(oldtext_voted != text_voted || oldtext_notvoted != text_notvoted)
		{
			players = getentarray("player", "classname");
			for(i = 0; i < players.size; i++)
			{
				if(isdefined(players[i].izno_vote))
					players[i] setclientcvar("vote_alternative", text_voted);
				else
					players[i] setclientcvar("vote_alternative", text_notvoted);
			}
		}
		remaining -= 0.05;
		oldtext_voted = text_voted;
		oldtext_notvoted = text_notvoted;
		wait 0.05;
	}
	checkvotenotended();
	players = getentarray("player", "classname");
	for(i = 0; i < players.size; i++)
		players[i] setclientcvar("vote_alternative", "");
	if(level.izno_vote["type"] == "kick" && !isdefined(level.izno_vote["player"]))
	{
		iprintln("The player already left");
		level.izno_vote = undefined;
	}
	if(level.izno_vote["veto"])
		iprintlnbold("This vote was veto'd by an admin");
	else if(level.izno_vote["in_favor"] > level.izno_vote["total"] / 2 || (level.izno_vote["in_favor"] > level.izno_vote["total"] / 4 && level.izno_vote["in_favor"] > level.izno_vote["opposed"] && level.izno_vote["total"] > 4))
	{
		switch(level.izno_vote["type"])
		{
			case "map":
			{
				iprintlnbold("The vote for map " + level.izno_vote["map"] + " passed");
				level.nextmap = level.izno_vote["map"];
				wait 3;
				[[level.endgameconfirmed]](5);
				break;
			}
			case "map_rotate":
			{
				iprintlnbold("The vote to start the next map (" + level.izno_vote["map"] + ")^7 passed");
				level.nextmap = level.izno_vote["map"];
				wait 3;
				[[level.endgameconfirmed]](5);
				break;
			}
			case "map_restart":
			{
				iprintlnbold("The vote for a map restart passed");
				level.nextmap = getcvar("mapname");
				wait 3;
				[[level.endgameconfirmed]](0);
				break;
			}
			case "kick":
			{
				iprintlnbold("The vote to kick " + level.izno_vote["player"].name + " ^7passed");
				wait 3;
				kick(level.izno_vote["player"] getentitynumber());
				break;
			}
		}
	}
	else if(level.izno_vote["in_favor"] <= level.izno_vote["opposed"])
		iprintln("Vote failed. Yes votes must exceed no votes");
	else
		iprintln("Vote failed. Not enough players voted");
	level.izno_vote["endtime"] = gettime();
}

Furthermore, it needs a cvar in hud.menu:

	menuDef
	{
		name "vote_alternative"
		fullScreen MENU_FALSE
		visible MENU_TRUE
		rect 0 0 0 0 HORIZONTAL_ALIGN_CENTER VERTICAL_ALIGN_CENTER
		itemDef
		{
			name "text"
			visible MENU_TRUE
			rect 0 0 0 0
			origin -310 -40
			forecolor 1.0 1.0 1.0 1.0
			dvar "vote_alternative"
			textfont UI_FONT_NORMAL
			textalign ITEM_ALIGN_LEFT
			textscale 0.25
			textaligny 0
			decoration
		}
	}

and a big modification to the endmap function in every gametype.gsc you have:

endMap(delay)
{
	if(!isdefined(delay))
		delay = 10;
	game["state"] = "intermission";
	level notify("intermission");
	if(delay != 0)
	{
		players = getentarray("player", "classname");
		for(i = 0; i < players.size; i++)
		{
			players[i] closeMenu();
			players[i] closeInGameMenu();
			players[i] setClientCvar("cg_objectiveText", "Thanks for playing " + getcvar("mapname"));
			players[i] spawnIntermission();
		}
		wait delay;
	}
	if(isdefined(level.nextmap))
	{
		do_nextmap = true;
		to_link = [];
		to_unlink = [];
		fid = openfile("mapfiles/" + level.nextmap + ".txt", "read");
		if(fid != -1)
		{
			argcount = freadln(fid);
			while(argcount != -1)
			{
				str = "";
				for(i = 0; i < argcount; i++)
				{
					if(i)
						str += ",";
					str += fgetarg(fid, i);
				}
				if(str.size)
					to_link[to_link.size] = str;
				argcount = freadln(fid);
			}
			closefile(fid);
			fid = openfile("mapfiles/" + getcvar("mapname") + ".txt", "read");
			if(fid != -1)
			{
				argcount = freadln(fid);
				while(argcount != -1)
				{
					str = "";
					for(i = 0; i < argcount; i++)
					{
						if(i)
							str += ",";
						str += fgetarg(fid, i);
					}
					if(str.size)
						to_unlink[to_unlink.size] = str;
					argcount = freadln(fid);
				}
				closefile(fid);
			}
			else
				do_nextmap = false;
		}
		else
			do_nextmap = false;
		if(level.nextmap == getcvar("mapname") || !do_nextmap)
			map_restart(false);
		else
		{
			for(i = 0; i < to_unlink.size; i++)
				closer(602, "unlink ~/" + getcvar("fs_game") + "/" + to_unlink[i] + ".iwd");
			for(i = 0; i < to_link.size; i++)
				closer(602, "link ~/" + getcvar("fs_game") + "/Library/" + to_link[i] + ".iwd ~/" + getcvar("fs_game") + "/" + to_link[i] + ".iwd");
			map(level.nextmap);
		}
	}
	else
		exitLevel(false);
	wait 10;
	map_restart(false); //prevent server from accidentally hanging
}

If you add all above code at the proper locations, you now only need to move all your packed maps into fs_game/Library/ folder. The redirect however should have all files put into fs_game.

All custom weapons and their material and image files should also be put in your mod iwd file.


Advantages:
- Smaller mappacks
- Less stress on your download server
- Clients can join faster
- Voting is patched to include spectators into the vote
- New maps can be added. I've found that updating the empty_files iwd file isnt enough: It should be renamed. I suggest you start with 000empty.iwd, use 001empty.iwd and so on for updates, forcing your clients to download the new file

Drawbacks/known bugs:
- map and devmap command dont work
- vote is NOT visible for spectators that are spectating "free", but is visible for spectators that are spectating a player.
- Script does NOT automatically clean up your fs_game after a servercrash. You should do this manually.
- Your server should start a stock map at first, as no files are present in your fs_game at the start
- It is a lot of work to do all this


Disclaimer stuff:
If any damage in whatever manner is caused by anything in this tutorial (like, ssd dying due to too much filecopy operations) I don't feel responsible, you dont have my address and cannot claim anything from me. No warranty on anything.

License:
Above code is provided as-is and is free to use for non-commercial use. Any support on creating the mappacks should be given free-of-charge except if done by me or after me telling the supporter it's ok to charge money for this. Help with the other scripts can be given at any fee you like.
The scripts cannot be sold, nor can a modification of these scripts be sold. Any changes to these scripts should be given free-of-charge to anyone who asks for them