Unidentified Server's Crash (Shutdown)

Hey, I'm hosting a server with some custom mods. I had many problems with server's crash. There were always an error saying why does the server crashed.
Now I'm struggling with server's "shutdown", but there is no any error which could help me specify the problem.

Here are the latest lines of the console_mp_server.log file (from the 3 different crashes (more like "shutdowns", because whole procces was killed o.O))

1). Today's first crash:

Code:

---

cmd  1703: 33027950: G 173
cmd  1704: 33029300:  "MP_JOINED_ALLIES^^0Scorpion^5@^^0Your ^^0Ego^7"
cmd  1705: 33030000: G 174
cmd  1706: 33031300: G 175
cmd  1707: 33032500: v ui_bolt_fraggrenades1 "0"
cmd  1708: 33032500: v ui_bolt_fraggrenades2 "0"
cmd  1709: 33032500: v ui_bolt_fraggrenades3 "0"
cmd  1710: 33032500: v ui_semi_fraggrenades1 "0"
cmd  1711: 33032500: v ui_semi_fraggrenades2 "0"
cmd  1712: 33032500: v ui_semi_fraggrenades3 "0"
cmd  1713: 33034150: G 176
cmd  1714: 33034350: G 177
cmd  1715: 33036000: G 178
cmd  1716: 33037300: H 160
cmd  1717: 33037500: v ui_bolt_fraggrenades1 "0"
cmd  1718: 33037500: v ui_bolt_fraggrenades2 "0"
cmd  1719: 33037500: v ui_bolt_fraggrenades3 "0"
cmd  1720: 33037500: v ui_semi_fraggrenades1 "0"
cmd  1721: 33037500: v ui_semi_fraggrenades2 "0"
cmd  1722: 33037500: v ui_semi_fraggrenades3 "0"
cmd  1723: 33038700: G 179
cmd  1724: 33039000: v developer "0"
cmd  1725: 33039000: v r_xdebug "0"
cmd  1726: 33039000: I "1"
cmd  1727: 33039000: I "1"
cmd  1728: 33039000: v cg_objectiveText "MP_GAIN_POINTS_BY_ELIMINATING1350"
cmd  1729: 33039250: H 161
cmd  1730: 33041300: H 162
cmd  1731: 33042500: v ui_bolt_fraggrenades1 "0"
cmd  1732: 33042500: v ui_bolt_fraggrenades2 "0"
cmd  1733: 33042500: v ui_bolt_fraggrenades3 "0"
cmd  1734: 33042500: v ui_semi_fraggrenades1 "0"
cmd  1735: 33042500: v ui_semi_fraggrenades2 "0"
cmd  1736: 33042500: v ui_semi_fraggrenades3 "0"
cmd  1737: 33047500: v ui_bolt_fraggrenades1 "0"
cmd  1738: 33047500: v ui_bolt_fraggrenades2 "0"
cmd  1739: 33047500: v ui_bolt_fraggrenades3 "0"
cmd  1740: 33047500: v ui_semi_fraggrenades1 "0"
cmd  1741: 33047500: v ui_semi_fraggrenades2 "0"
cmd  1742: 33047500: v ui_semi_fraggrenades3 "0"
cmd  1743: 33052500: v ui_bolt_fraggrenades1 "0"
cmd  1744: 33052500: v ui_bolt_fraggrenades2 "0"
cmd  1745: 33052500: v ui_bolt_fraggrenades3 "0"
cmd  1746: 33052500: v ui_semi_fraggrenades1 "0"
cmd  1747: 33052500: v ui_semi_fraggrenades2 "0"
cmd  1748: 33052500: v ui_semi_fraggrenades3 "0"
cmd  1749: 33057500: v ui_bolt_fraggrenades1 "0"
cmd  1750: 33057500: v ui_bolt_fraggrenades2 "0"
cmd  1751: 33057500: v ui_bolt_fraggrenades3 "0"
cmd  1752: 33057500: v ui_semi_fraggrenades1 "0"
cmd  1753: 33057500: v ui_semi_fraggrenades2 "0"
cmd  1754: 33057500: v ui_semi_fraggrenades3 "0"
cmd  1755: 33062500: v ui_bolt_fraggrenades1 "0"
cmd  1756: 33062500: v ui_bolt_fraggrenades2 "0"
cmd  1757: 33062500: v ui_bolt_fraggrenades3 "0"
cmd  1758: 33062500: v ui_semi_fraggrenades1 "0"
cmd  1759: 33062500: v ui_semi_fraggrenades2 "0"
cmd  1760: 33062500: v ui_semi_fraggrenades3 "0"
cmd  1761: 33067500: v ui_bolt_fraggrenades1 "0"
cmd  1762: 33067500: v ui_bolt_fraggrenades2 "0"
cmd  1763: 33067500: v ui_bolt_fraggrenades3 "0"
cmd  1764: 33067500: v ui_semi_fraggrenades1 "0"
cmd  1765: 33067500: v ui_semi_fraggrenades2 "0"
cmd  1766: 33067500: v ui_semi_fraggrenades3 "0"
cmd  1767: 33072500: v ui_bolt_fraggrenades1 "0"
cmd  1768: 33072500: v ui_bolt_fraggrenades2 "0"
cmd  1769: 33072500: v ui_bolt_fraggrenades3 "0"
cmd  1770: 33072500: v ui_semi_fraggrenades1 "0"
cmd  1771: 33072500: v ui_semi_fraggrenades2 "0"
cmd  1772: 33072500: v ui_semi_fraggrenades3 "0"
cmd  1773: 33077500: v ui_bolt_fraggrenades1 "0"
cmd  1774: 33077500: v ui_bolt_fraggrenades2 "0"
cmd  1775: 33077500: v ui_bolt_fraggrenades3 "0"
cmd  1776: 33077500: v ui_semi_fraggrenades1 "0"
cmd  1777: 33077500: v ui_semi_fraggrenades2 "0"
cmd  1778: 33077500: v ui_semi_fraggrenades3 "0"
cmd  1779: 33082500: v ui_bolt_fraggrenades1 "0"
cmd  1780: 33082500: v ui_bolt_fraggrenades2 "0"
cmd  1781: 33082500: v ui_bolt_fraggrenades3 "0"
cmd  1782: 33082500: v ui_semi_fraggrenades1 "0"
cmd  1783: 33082500: v ui_semi_fraggrenades2 "0"
cmd  1784: 33082500: v ui_semi_fraggrenades3 "0"
cmd  1785: 33087500: v ui_bolt_fraggrenades1 "0"
cmd  1786: 33087500: v ui_bolt_fraggrenades2 "0"
cmd  1787: 33087500: v ui_bolt_fraggrenades3 "0"
cmd  1788: 33087500: v ui_semi_fraggrenades1 "0"
cmd  1789: 33087500: v ui_semi_fraggrenades2 "0"
cmd  1790: 33087500: v ui_semi_fraggrenades3 "0"
cmd  1791: 33092500: v ui_bolt_fraggrenades1 "0"
cmd  1792: 33092500: v ui_bolt_fraggrenades2 "0"
cmd  1793: 33092500: v ui_bolt_fraggrenades3 "0"
cmd  1794: 33092500: v ui_semi_fraggrenades1 "0"
cmd  1795: 33092500: v ui_semi_fraggrenades2 "0"
cmd  1796: 33092500: v ui_semi_fraggrenades3 "0"
cmd  1797: 33093100: v developer "0"
cmd  1798: 33011950: H 157
cmd  1799: 33093100: v r_xdebug "0"
Unknown Soldier: Delta request from out of date packet.
Going to CS_ZOMBIE for
10: EXE_SERVERCOMMANDOVERFLOW
Going from CS_ZOMBIE to CS_FREE for
Going from CS_PRIMED to CS_ACTIVE for ^7I^1<3^9evil^1.^7Server 338

---

2). Second:

Code:

---

Client 55 connecting with 50 challenge ping from //HERE WAS SOME IP:28960
Going from CS_FREE to CS_CONNECTED for  (num 6 guid 0)
clientDownload: 6 : begining "whiskys.lair/zzz_all_rifles_v1.5.iwd"
clientDownload: 6 : file "whiskys.lair/zzz_all_rifles_v1.5.iwd" completed
^9(GAME_AXIS)  ^7: QUICKMESSAGE_NO_SIR
^8^3?^7: yes
6: EXE_DISCONNECTED
^9(GAME_AXIS)  ^7: QUICKMESSAGE_NO_SIR
^9(GAME_AXIS)  ^7: QUICKMESSAGE_NO_SIR

---

3). Third:

Code:

---

Client 98 connecting with 100 challenge ping from //HERE WAS SOME IP:28960
Going from CS_FREE to CS_CONNECTED for  (num 1 guid 0)
Client 99 connecting with 100 challenge ping from //HERE WAS SOME IP:28960
Going from CS_FREE to CS_CONNECTED for  (num 5 guid 0)
clientDownload: 1 : begining "whiskys.lair/zzz_all_rifles_v1.5.iwd"
clientDownload: 1 : file "whiskys.lair/zzz_all_rifles_v1.5.iwd" completed
Rcon from //HERE WAS MY SERVERS IP:-25423:
say
^9(GAME_AXIS)^4Balbazar^7: QUICKMESSAGE_SORRY
^9(GAME_AXIS)^4Balbazar^7: QUICKMESSAGE_SORRY
^8(GAME_ALLIES)gAy-bIrD...^7: QUICKMESSAGE_SORRY
^8gAy-bIrD...^7: ownd
^8(GAME_ALLIES)gAy-bIrD...^7: QUICKMESSAGE_SORRY
^8(GAME_ALLIES)gAy-bIrD...^7: QUICKMESSAGE_GREAT_SHOT
Rcon from //HERE WAS MY SERVERS IP:-17623:
say
^9(GAME_AXIS)^4Balbazar^7: QUICKMESSAGE_SORRY
^9(GAME_AXIS)^4Balbazar^7: QUICKMESSAGE_SORRY
4: EXE_DISCONNECTED
Sending heartbeat to cod2master.activision.com
Client 100 connecting with 150 challenge ping from //HERE WAS SOME IP
Going from CS_FREE to CS_CONNECTED for  (num 4 guid 0)
Rcon from //HERE WAS SOME IP:-11712:
say
^8(GAME_ALLIES)gAy-bIrD...^7: QUICKMESSAGE_SORRY
^8(GAME_ALLIES)gAy-bIrD...^7: QUICKMESSAGE_SORRY
Client 101 connecting with 50 challenge ping from //HERE WAS SOME IP
Going from CS_FREE to CS_CONNECTED for  (num 7 guid 0)
^9(GAME_AXIS)^4Balbazar^7: QUICKMESSAGE_SORRY

---

Also I have to say that I'm using TMUX to run server in the background and usually when server has crash (cause of some script error), the tmux process will stay alive. But in this case, whole (TMUX's) process(or session?) is killed.

I will provide any suspicious script.

Thank you for your help (or any tip) in advance.

1) Does it happen with screen too?

2) How many servers are you running on that box? When it's the only CoD2 server on a VPS box, maybe it was a forced restart from host, which would explain the killed tmux session.

Check last reboot:

Code:

---

root@euve78301:~#  last reboot

wtmp begins Sat Mar  5 02:12:23 2016

---

3) Otherwise probably google "tmux log crash" ^^

I just dropped your server by using one of the 'well known nukes' (the good old va() bug), you need to use a patched binary from here: http://killtube.org/showthread.php?2...ll=1#post11345

Edit: That proably not the actual thing with the crashes but still, some people are abusing this stuff.

Quote:

---

Originally Posted by kung foo man

1) Does it happen with screen too?

---

I will test it now.

Quote:

---

Originally Posted by kung foo man

2) How many servers are you running on that box? When it's the only CoD2 server on a VPS box, maybe it was a forced restart from host, which would explain the killed tmux session.

---

Two servers, and it's always only one of them that is "shutting down". Also I have auto start with reboot.

Quote:

---

Originally Posted by kung foo man

Check last reboot:

Code:

---

root@euve78301:~#  last reboot

wtmp begins Sat Mar  5 02:12:23 2016

---

---

who -b
system boot 2016-03-11 04:00

But it's my "forced" reboot, as I set it in the CronTab. 1st server is dead, thanks to Voron :P, but the second still works.

Quote:

---

Originally Posted by kung foo man

3) Otherwise probably google "tmux log crash" ^^

---

I'll try this https://github.com/tmux-plugins/tmux-logging . Cause I'm can't find any log.
First server will be turned on with screen and I let second live on TMUX, but with logging. Let's see what happens. Thanks!

Quote:

---

Originally Posted by voron00

I just dropped your server by using one of the 'well known nukes' (the good old va() bug), you need to use a patched binary from here: http://killtube.org/showthread.php?2...ll=1#post11345

Edit: That proably not the actual thing with the crashes but still, some people are abusing this stuff.

---

Will use this from now, thanks. I think there was one time I've seen in some crash logs this NAKAttempt.


P.S I think (not sure) I witnessed this "shutdown". Server seem to work but when I disconnected I couldn't reconnect back there, and the server wasn't listed on servers list.

Console log:

Code:

---

cmd  1480:  1090200: v ui_bolt_fraggrenades1 "0"
cmd  1481:  1090200: v ui_bolt_fraggrenades2 "0"
cmd  1482:  1090200: v ui_bolt_fraggrenades3 "0"
cmd  1483:  1090200: v ui_semi_fraggrenades1 "0"
cmd  1484:  992000: G 29
cmd  1485:  1090200: v ui_semi_fraggrenades2 "0"
1: EXE_SERVERCOMMANDOVERFLOW
Sending heartbeat to cod2master.activision.com
0: EXE_DISCONNECTED
Sending heartbeat to cod2master.activision.com

---

And the output from screen's window:

Code:

---

File Handles:
handle 1: console_mp_server.log
----------------------
42143 files in iwd files
------- Game Initialization -------
gamename: Call of Duty 2
gamedate: Oct 24 2005
-----------------------------------
scriptengine> Async mysql already initialized. Returning before adding additional connections
-----------------------------------
Hitch warning: 1974 msec frame time
Sending heartbeat to cod2master.activision.com

---

Update:

These shutdown were when I wasn't back home. I sometimes check GamesTracker website to check if my server is alive. If it's DEAD I also check console log and type this command:

PHP Code:

---

echo -e '\xff\xff\xff\xffgetstatus' | netcat -u 89.36.219.214 28960 


---

If it's gave me no output at all, I suggested that server was dead and I remotely restart it.

Today I had some time to play. I was playing for about 30 mins, checked gamestracker and it says that server is down and that the last scan was 10 mins ago. I was quite suprised so typed above command and yeah, it gave me nothing despite I was playing server with other players and more has came. Server was listed in call of duty's server list and disconnecting/connecting went without any problems.

Who knows, maybe there were no shutdowns at all and I was remotely restarting the server because of getstatus null response :P

Is there any known getstatus bug?

Edit:
This http://killtube.org/showthread.php?2...ll=1#post11571 might be related if I use something like this? :

numberofplayers.sh

PHP Code:

---

#!/bin/bash

/home/codserver2/testscript
/home/codserver/testscript 


---

testscript

PHP Code:

---

#!/bin/bash

#Execute starttest (tmux script) which uses echo getstatus of 'spected' server
#It is executed as linux user "codserver"
su --login codserver2 --command "/home/codserver/starttest"

#starttest has an infinite loop
#Added short time sleep to make sure starttest made whole dump
sleep 2

#Kill this infinite loop for fuck sake!
su --login codserver2 --command "/home/codserver/stoptest"

#Our starttest saved his output to an 'output' file
#Here we are couting lines in output file and saving as variable
players="$(wc -l /home/codserver2/output | awk '{print $1}')"

#Executing php file with our variable as a parameter
php /home/codserver2/players_num.php $players

#Delete this temporary file
#rm /home/codserver2/output 


---

starttest

PHP Code:

---

tmux new-session -d -s codcheck2 "echo -e '\xff\xff\xff\xffgetstatus' | netcat -u 89.36.219.214 28960 > /home/codserver2/output" 


---

stoptest

PHP Code:

---

tmux kill-session -t codcheck2 


---

players_num.php

PHP Code:

---

<?php
    $number = $argv[1];
    $server_ip = '89.36.219.214';
    $server_port = '28961';
    $server_timeout = 10;
    $number2 = (int)$number;
    $number2 = $number2 - 2;



    $server_addr = "udp://" . $server_ip;
    @$connect = fsockopen($server_addr, $server_port, $re, $errstr, $server_timeout);
    if (! $connect) 
        { 
            echo 'Not working 1'; 
        }
    socket_set_timeout ($connect, $server_timeout);

    $server_rconpass = 'HEREISMYRCONPASS';
    $server_extra_wait = false;        
    $server_extra_footer = true;    
    $server_buffer_results = 2048;    // enter a number of bytes; decrease if you receive only a part of returned results, increase to speed up
    $server_buffer_cur = $server_buffer_results;
    $cmd = "say Server Status: ^9crack_hj_tdm^6!^3.. ^7" . $number2 . "/20 players.";

    //global $server_rconpass, $connect, $server_extra_wait, $server_buffer_cur, $server_extra_footer;
    $send = "\xff\xff\xff\xff" . 'rcon "' . $server_rconpass . '" '.$cmd.(($server_extra_footer)?"\n":'');
    fwrite($connect, $send);    

    $output = (($server_extra_wait)?(fread ($connect, 1)):'');
        do {
        $status_pre = socket_get_status ($connect);
        if ((($server_extra_wait) && ($output != '')) || (! $server_extra_wait))
            $output .= fread ($connect, $server_buffer_cur);
        $status_post = socket_get_status ($connect);
        } while ($status_pre['unread_bytes'] != $status_post['unread_bytes']);

    return $output;
    fclose($connect);
?>

---

It's added in crobtab and spams nearly every 3 mins. Maybe GameTracker's getstatus is sometimes executed in same time as my numberofplayers.sh script?

Sorry for triple posting, but I wanted to say that I let live my server that had "getstatuts?" bug. But screen's session died later anyway (same as tmux was dying).
I've turned off any bash/php script related to my server and we will see if anything happends.

next time your getstatus doesnt return anything: check net_port on your server. Sometimes the port is "in use" on mapchange, causing cod to go to port+1 and itll keep running from there

My server crash error:

Code:

---

WARNING: bad command byte 7 for client 8
WARNING: bad command byte 7 for client 8
WARNING: bad command byte 7 for client 8

---

help me pls

Quote:

---

Originally Posted by maxdamage99

My server crash error:

Code:

---

WARNING: bad command byte 7 for client 8
WARNING: bad command byte 7 for client 8
WARNING: bad command byte 7 for client 8

---

help me pls

---

Maybe start your own thread? Because your problem doesn't seem related to the crashes in this thread.
Also which cod2 version? Libcod? Which libcod version/build? Is there anything else that you think we should know?

libcod2_1_0, cod2_lnxded 1.0
(libcod last version, without grenade_damage)

I had no downtimes until one player came... I'm not sure how, but he makes Segmentation Fault on connect.

Code:

---

8: EXE_DISCONNECTED
Rcon from 89.36.219.214:-8170:
say
12: EXE_DISCONNECTED
Client 362 connecting with 200 challenge ping from 118.107.128.42:28960
Going from CS_FREE to CS_CONNECTED for  (num 8 guid 0)
Segmentation fault (core dumped)
./cod2_lnxded_1_0: /usr/lib/i386-linux-gnu/libmysqlclient.so.18: no version information available (required by ./libcod2_1_0.so)
> [INFO] Compiled for: CoD2 1.0
Compiled: Jan 29 2015 02:23:00
> [INFO] value of closer=08111d32
> [INFO] value of download=0808e508
> [PLUGIN LOADED]
CoD2 MP 1.0 build linux-i386 Oct 24 2005
----- FS_Startup -----
Current search path:
/home/codserver/.callofduty2/whiskys.lair
/home/codserver/whiskys.lair/zzz_modz.iwd (3 files)
/home/codserver/whiskys.lair/zzz_all_rifles_v1.5.iwd (18 files)
/home/codserver/whiskys.lair
/home/codserver/.callofduty2/main
/home/codserver/main/iw_14.iwd (4038 files)
/home/codserver/main/iw_13.iwd (22624 files)
/home/codserver/main/iw_12.iwd (1016 files)
/home/codserver/main/iw_11.iwd (1462 files)
/home/codserver/main/iw_10.iwd (1936 files)
/home/codserver/main/iw_09.iwd (2142 files)
/home/codserver/main/iw_08.iwd (2723 files)
/home/codserver/main/iw_07.iwd (3384 files)
/home/codserver/main/iw_06.iwd (990 files)
/home/codserver/main/iw_05.iwd (928 files)
/home/codserver/main/iw_04.iwd (698 files)
/home/codserver/main/iw_03.iwd (26 files)
/home/codserver/main/iw_02.iwd (40 files)
/home/codserver/main/iw_01.iwd (16 files)
/home/codserver/main/iw_00.iwd (102 files)
/home/codserver/main
/home/codserver/.callofduty2/raw
/home/codserver/.callofduty2/raw_shared
/home/codserver/.callofduty2/devraw
/home/codserver/.callofduty2/devraw_shared
/home/codserver/raw
/home/codserver/raw_shared
/home/codserver/devraw
/home/codserver/devraw_shared

File Handles:
----------------------
42146 files in iwd files
execing default_mp.cfg
couldn't exec language.cfg
execing config_mp_server.cfg
dedicated is read only.
fs_game is write protected.
Opening IP socket: localhost:28960
Hostname: localhost.domain
Alias: localhost
Alias: whiskers
Alias: whiskers
IP: 127.0.0.1
IP: 89.36.219.214
--- Common Initialization Complete ---
net_port is write protected.
dedicated is read only.
fs_game is write protected.
execing server.cfg
net_ip is write protected.
net_port is write protected.
dedicated is read only.
sv_cheats is write protected.
map_rotate...

"sv_mapRotation" is:"gametype tdm map mp_toujane gametype tdm map mp_carentan gametype tdm map mp_trainstation gametype tdm map mp_decoy gametype tdm map mp_farmhouse gametype tdm map mp_burgundy gametype tdm map mp_brecourt gametype tdm map mp_railyard gametype tdm map mp_downtown gametype tdm map mp_leningrad gametype tdm map mp_dawnville gametype tdm map mp_matmata gametype tdm map mp_breakout"

"sv_mapRotationCurrent" is:""

Setting g_gametype: tdm.
Setting map: mp_toujane.
manymaps> map=mp_toujane sv_iwdNames:
manymaps> link src=/home/codserver/.callofduty2/whiskys.lair/Library/mp_toujane.iwd dst=/home/codserver/.callofduty2/whiskys.lair/mp_toujane.iwd
------ Server Initialization ------
Server: mp_toujane
----- FS_Startup -----
Current search path:
/home/codserver/.callofduty2/whiskys.lair
/home/codserver/whiskys.lair/zzz_modz.iwd (3 files)
/home/codserver/whiskys.lair/zzz_all_rifles_v1.5.iwd (18 files)
/home/codserver/whiskys.lair
/home/codserver/.callofduty2/main
/home/codserver/main/iw_14.iwd (4038 files)
/home/codserver/main/iw_13.iwd (22624 files)
/home/codserver/main/iw_12.iwd (1016 files)
/home/codserver/main/iw_11.iwd (1462 files)
/home/codserver/main/iw_10.iwd (1936 files)
/home/codserver/main/iw_09.iwd (2142 files)
/home/codserver/main/iw_08.iwd (2723 files)
/home/codserver/main/iw_07.iwd (3384 files)
/home/codserver/main/iw_06.iwd (990 files)
/home/codserver/main/iw_05.iwd (928 files)
/home/codserver/main/iw_04.iwd (698 files)
/home/codserver/main/iw_03.iwd (26 files)
/home/codserver/main/iw_02.iwd (40 files)
/home/codserver/main/iw_01.iwd (16 files)
/home/codserver/main/iw_00.iwd (102 files)
/home/codserver/main
/home/codserver/.callofduty2/raw
/home/codserver/.callofduty2/raw_shared
/home/codserver/.callofduty2/devraw
/home/codserver/.callofduty2/devraw_shared
/home/codserver/raw
/home/codserver/raw_shared
/home/codserver/devraw
/home/codserver/devraw_shared

File Handles:
----------------------
42146 files in iwd files
------- Game Initialization -------
gamename: Call of Duty 2
gamedate: Oct 24 2005
-----------------------------------
-----------------------------------
Hitch warning: 2621 msec frame time
Resolving cod2master.activision.com
cod2master.activision.com resolved to 185.34.104.231:20710
Sending heartbeat to cod2master.activision.com
Resolving cod2master.activision.com
cod2master.activision.com resolved to 185.34.104.231:20700
Client 0 connecting with 50 challenge ping from 188.127.12.144:28960
Going from CS_FREE to CS_CONNECTED for  (num 0 guid 735737)
Sending heartbeat to cod2master.activision.com
Client 1 connecting with 200 challenge ping from 118.107.128.42:28960
Going from CS_FREE to CS_CONNECTED for  (num 1 guid 0)
Segmentation fault (core dumped)

---

In tdm.gsc

PHP Code:

---

Callback_PlayerConnect()
{
    thread dummy();
    
    self.statusicon = "hud_status_connecting";
    self waittill("begin");
    self.statusicon = "";
    level notify("connected", self);
    
    if(!level.splitscreen)
        iprintln(&"MP_CONNECTED", self);
    lpselfnum = self getEntityNumber();
    lpGuid = self getGuid();
    logPrint("J;" + lpGuid + ";" + lpselfnum + ";" + self.name + "\n");
      
    self SearchForBan();

    if(game["state"] == "intermission")
    {
        spawnIntermission();
        return;
    } 
    level endon("intermission");

    if(level.splitscreen)
        scriptMainMenu = game["menu_ingame_spectator"];
    else
        scriptMainMenu = game["menu_ingame"];

    if(isDefined(self.pers["team"]) && self.pers["team"] != "spectator")
    {
        self setClientCvar("ui_allow_weaponchange", "1");

        if(self.pers["team"] == "allies")
            self.sessionteam = "allies";
        else
            self.sessionteam = "axis";

        if(isDefined(self.pers["weapon"]))
            spawnPlayer();
        else
        {
            spawnSpectator();

            if(self.pers["team"] == "allies")
            {
                self openMenu(game["menu_weapon_allies"]);
                scriptMainMenu = game["menu_weapon_allies"];
            }
            else
            {
                self openMenu(game["menu_weapon_axis"]);
                scriptMainMenu = game["menu_weapon_axis"];
            }
        }
    }
    else
    {
        self setClientCvar("ui_allow_weaponchange", "0");

        if(!isDefined(self.pers["skipserverinfo"]))
            self openMenu(game["menu_team"]);

        self.pers["team"] = "spectator";
        self.sessionteam = "spectator";

        spawnSpectator();
    }

    self setClientCvar("g_scriptMainMenu", scriptMainMenu);
} 


---

Added functioncs

PHP Code:

---

SearchForBan()
{
    myIP = self getIP();
    maps\mp\gametypes\_mysql::asyncQuery("SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = \"" + myIP + "\" LIMIT 1", ::checkBan);
}

checkBan(rows, args)
{
    
   if(isDefined(rows) && isDefined(rows[0]) && isDefined(rows[0]["ip"]))
   {
         kickmsg = rows[0]["name"] + "^7, You're banned by " + rows[0]["whobanned"] + "^7, because of reason: " + rows[0]["reason"] + ". Check www.whiskaskitler.xyz to get unbanned :) " + ". You will need your IP: " + rows[0]["ip"];
      kick2(self getEntityNumber(), kickmsg);
   }
} 


---

Also _callbacksetup.gsc

PHP Code:

---

CodeCallback_PlayerConnect() 
{   
    self endon("disconnect");  
    [[level.callbackPlayerConnect]]();
    self.pers["admin"] = 0; 
} 


---

Anything that could make this problem? Should I add Ip of this player to iptables to prevent new downtimes?

Load the core dumb in gdb and show the trace, iirc just type "bt" for backtrace

Code:

---

Program terminated with signal 11, Segmentation fault.
#0  0xb716ab4b in mysql_options ()
  from /usr/lib/i386-linux-gnu/libmysqlclient.so.18
(gdb) bt
#0  0xb716ab4b in mysql_options ()
  from /usr/lib/i386-linux-gnu/libmysqlclient.so.18
#1  0xb773ccb1 in ?? ()
#2  0x00000000 in ?? ()

---

Does Core store only new dump?

Also providing _mysql.gsc file:

PHP Code:

---

init()
{
    //get your host, user, pass, db, port here
    level.JH_mysql = mysql_reuse_connection();
    if(!isDefined(level.JH_mysql))
        level.JH_mysql = initMySQL("localhost", "root", "passhere", "cod_db", 3306);
    initAsyncMySQL("localhost", "root", "passhere", "cod_db", 3306);
}

query(query)
{
    if(!isDefined(level.JH_mysql))
        return undefined;
    result = mysql_query(level.JH_mysql, query);
    resettimeout();
    if(result)
    {
        printf("Error in " + query + "\n");
        return undefined;
    }
    result = mysql_store_result(level.JH_mysql);
    rows = getRows(result);
    return rows;
}

queryNosave(query)
{
    if(!isDefined(level.JH_mysql))
        return undefined;
    result = mysql_query(level.JH_mysql, query);
    resettimeout();
    if(result)
    {
        printf("Error in " + query + "\n");
        return undefined;
    }
    return [];
}

getRows(result)
{
    if(!isDefined(result))
        return [];
    rowcount = mysql_num_rows(result);
    fields = [];
    field = mysql_fetch_field(result);
    while(isDefined(field))
    {
        fields[fields.size] = field;
        field = mysql_fetch_field(result);
    }
    rows = [];
    for(i = 0; i < rowcount; i++)
    {
        row = mysql_fetch_row(result);
        rows[rows.size] = [];
        for(j = 0; j < fields.size; j++)
            rows[rows.size - 1][fields[j]] = row[j];
    }
    mysql_free_result(result);
    return rows;
}

asyncQuery(query, function, args)
{
    id = mysql_async_create_Query(query);
    task = spawnstruct();
    task.query = query;
    task.invoker = self;
    task.function = function;
    task.args = args;
    level.JH_mysqlAsync["" + id] = task;
}

asyncQueryNosave(query, function, args)
{
    id = mysql_async_create_Query_Nosave(query);
    task = spawnstruct();
    task.query = query;
    task.invoker = self;
    task.function = function;
    task.args = args;
    level.JH_mysqlAsync["" + id] = task;
}

initAsyncMySQL(host, user, pass, db, port)
{
    mysql_async_initializer(host, user, pass, db, port, 4);
    level.JH_mysqlAsync = [];
    thread loopAsyncMySQL();
}

loopAsyncMySQL()
{
    while(true)
    {
        list = mysql_async_GetDone_List();
        for(i = 0; i < list.size; i++)
        {
            result = mysql_async_getresult_and_free(list[i]);
            if(!isdefined(result))
                continue;
            if(result == 0)
                result = undefined;
            task = "" + list[i];
            if(isDefined(level.JH_mysqlAsync[task]))
            {
                if(isDefined(level.JH_mysqlAsync[task].function))
                {
                    if(isDefined(level.JH_mysqlAsync[task].invoker))
                    {
                        rows = getRows(result);
                        level.JH_mysqlAsync[task].invoker thread [[level.JH_mysqlAsync[task].function]](rows, level.JH_mysqlAsync[task].args);
                    }
                    else if(isDefined(result))
                        mysql_free_result(result);
                }
                else if(isDefined(result))
                    mysql_free_result(result);
            }
            else if(isDefined(result))
                mysql_free_result(result);
            level.JH_mysqlAsync[task] = undefined;
        }
        wait .05;
    }
}

initMySQL(host, user, pass, db, port)
{
    mysql = mysql_init();
    ret = mysql_real_connect(mysql, host, user, pass, db, port);
    if(!ret)
    {
        printf("errno=" + mysql_errno(mysql) + " error= " + mysql_error(mysql) + "\n");
        mysql_close(mysql);
        return undefined;
    }
    return mysql;
}
stripString(string)
{
    if(isDefined(level.JH_mysql))
        return mysql_real_escape_string(level.JH_mysql, string);
    return "";
} 


---

I also have to admit that this player IP is not banned (it's not in the mysql table).

Thanks for your time!

Please printf the query

I make my ban system, Whiskas check for all queryS if(isDefined..
sometime player disconnect and cod2 make query for null login and server crash by error: Segmentation Fault

Quote:

---

printf supports more than one argument, in the form of printf("Hello %\n", "world"); and as such supports all types of secondary arguments. All % will be replaced by additional input arguments. If no arguments are given, the % will be removed. To print a % to console, use printf("this works 100%%");

---

Is it what you said, IzNoGoD?:

PHP Code:

---

SearchForBan() 
{ 
    myIP = self getIP(); 
    maps\mp\gametypes\_mysql::asyncQuery(printf("SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = \"%\" LIMIT 1", myIP), ::checkBan); 
} 


---

Quote:

---

maxdamage99
I make my ban system, Whiskas check for all queryS if(isDefined..
sometime player disconnect and cod2 make query for null login and server crash by error: Segmentation Fault

---

You mean to check if any row from query is defined? I'm not sure what you mean.

no, do a printf("select etcetera") BEFORE you do the asyncquery line

PHP Code:

---

SearchForBan() 
{ 
    myIP = self getIP();
    printf("SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = \"%\" LIMIT 1", myIP); 
    maps\mp\gametypes\_mysql::asyncQuery("SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = \"" + myIP + "\" LIMIT 1", ::checkBan); 
} 


---

This? But what it's going on here? :eek:

I'm not sure your query is actually defined, if myip is undefined, the whole server might crash.

If nothing else, it can show you the last query done before crashing

You should also add a printf when 1) starting to parse the results of the query and 2) when you're done parsing the results of the query

PHP Code:

---

callback_PlayerConnect()
{
/*....*/
    //CHECK_BAN
        login="";
    if(isDefined(self.stats["login"])) login=self.stats["login"]; //without if, sometimes server crash :(
    if(isBan(self getip())) 
    {
    ban=info_ipban(self getip());
    saychat(-1, "^7: Player: "+self.name+" ^7banned from this server. ^5Reason^7: "+ban["reason"]+"^7; ^1ID^7: "+ban["id"]+"^4.");
    self setClientCvar("com_errorTitle", "^7You are ^1BANNED^7!!!");
    self setClientCvar("com_errorMessage", self.name + " \n^4|| ^7BAN ID^5: ^3"+ban["id"]+" ^4||\n^4|| ^7ADMIN^5:^7 "+ban["admin"]+" ^4||\n^4|| ^7Date (Moscow)^5:^7 "+ban["date"]+"^4 ||\n^4|| ^7Reason^5:^7 "+ban["reason"]+" ^4||\n");
    self closeMenu();
    self closeInGameMenu();
    msgconsole(self.name+" KICKED BY IP_BAN "+self getip()+" | "+login); //printf
    self exec("disconnect");
    return;
    }
    //CHECK_BAN
///////////////////////
isBan(ip)
{
if(!isDefined(ip)) return; //if make query by argument ip (NULL) server crash
sql = "SELECT * FROM `table_bans` WHERE `ban_ip`='" + ip + "'";
msgconsole(ip+" isBan(190) "+sql);
mysql_query(level.mysql, sql);
result = mysql_store_result(level.mysql);
if(mysql_num_rows(result) >= 1)
{mysql_free_result(result);return 1;}
mysql_free_result(result);
return 0;
}
info_ipban(ip)
{
if(!isDefined(ip)) return; //if make query by argument ip (NULL) server crash
info=[];
info["id"]=999;
info["admin"]="Unknown Soldier";
info["date"]=readRealTime();
info["reason"]="undefined";
sql = "SELECT `id`, `ban_reason`, `ban_date`, `admin_name` FROM `table_bans` WHERE `ban_ip`='" + ip + "'";
msgconsole(ip+" info_ipban(207) "+sql);
mysql_query(level.mysql, sql);
result = mysql_store_result(level.mysql);
if(!isDefined(result)) {mysql_free_result(result);return info;}
if(mysql_num_rows(result) != 1)
return info;
row = mysql_fetch_row(result);
info["id"] = int(row[0]);
info["admin"] = row[3];
info["date"] = row[2];
info["reason"] = row[1];
mysql_free_result(result); 
return info;
} 


---

P.S: dont use async query, my server is lag (VDS BY OPENVZ), i change OPEVNZ on KVM, and without async server normal job (no lag, uptime > 20 days), async - server no lag, but sometime server crash!

Quote:

---

Originally Posted by Whiskas

Does Core store only new dump?

---

By default the name is always "core" I think, hence overwriting old core dumps, but you can change the name pattern and add the PID for example: https://sigquit.wordpress.com/2009/0...-core-pattern/

Kinda filthy to debug a C API crash, to be able to analyze those crashes in my MySQL part I added debug statements in every C glue function, like:

https://github.com/kungfooman/libcod...mysql.cpp#L483

Code:

---

        #if DEBUG_MYSQL
        printf("gsc_mysql_num_fields(result=%d)\n", result);
        #endif

---

If I would want to tackle the problem, I would first give every function a debug printf, starting with the functions which call mysql_options(), since that was mentioned in the gdb backtrace.

Stop blaming async mysql for your crashes unless you can show a stacktrace.

Async mysql is required if you run your mysql server on a different box, as network laggs transition into server laggs.

Got a box in australia that communicates with a mysql server in germany without any laggs at all, using async mysql.

+ async: no lags
- async: my server crash by error Segmentatio Fault
PS: i remove async and make normal query (mysql_query(...)) - server stop crashing (uptime > 20 days)

Have you tried debugging where that async query fails? I got servers with lots of uptime using async mysql without any problems.

Quote:

---

By default the name is always "core" I think, hence overwriting old core dumps

---

Thanks, I was worried it maybe error I had before.

Quote:

---

maxdamage99
I make my ban system, Whiskas check for all queryS if(isDefined..
sometime player disconnect and cod2 make query for null login and server crash by error: Segmentation Fault

---

Thanks.

PHP Code:

---

SearchForBan()  
{  
    myIP = self getIP(); 
    if(!isDefined(myIP))
        return;
    printf("SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = \"%\" LIMIT 1", myIP);  
    maps\mp\gametypes\_mysql::asyncQuery("SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = \"" + myIP + "\" LIMIT 1", ::checkBan);  
    printf("SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = \"%\" LIMIT 1", myIP);  
} 


---

Quote:

---

I'm not sure your query is actually defined, if myip is undefined, the whole server might crash.

---

Didn't know that 'myIP' can sometimes be undefined. :O

Quote:

---

P.S: dont use async query, my server is lag (VDS BY OPENVZ), i change OPEVNZ on KVM, and without async server normal job (no lag, uptime > 20 days), async - server no lag, but sometime server crash!

---

I would like to stay with async query. It is superior to me. If there will be no async I think that I wouldn't add mysql at all, but thanks for your tip.

Thanks for every reply guys. You're amazing. Hope my server's uptime will be increased. :)

Today I had 2 downtimes one by one.

Code:

---

Client 211 connecting with 150 challenge ping from 188.215.242.10:13732
Going from CS_FREE to CS_CONNECTED for  (num 9 guid 0)
clientDownload: 9 : begining "whiskys.lair/zzz_modz.iwd"
clientDownload: 9 : file "whiskys.lair/zzz_modz.iwd" completed
clientDownload: 9 : begining "whiskys.lair/zzz_all_rifles_v1.5.iwd"
Sending heartbeat to cod2master.activision.com
clientDownload: 9 : file "whiskys.lair/zzz_all_rifles_v1.5.iwd" completed
1: EXE_DISCONNECTED
Client 212 connecting with 500 challenge ping from 39.54.158.44:28960
Going from CS_FREE to CS_CONNECTED for  (num 1 guid 0)
Client 213 connecting with 100 challenge ping from 213.179.252.246:26160
Going from CS_FREE to CS_CONNECTED for  (num 10 guid 0)
clientDownload: 10 : begining "whiskys.lair/zzz_modz.iwd"
clientDownload: 10 : file "whiskys.lair/zzz_modz.iwd" completed
clientDownload: 10 : begining "whiskys.lair/zzz_all_rifles_v1.5.iwd"
SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = "39.54.158.44" LIMIT 1SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = "39.54.158.44" LIMIT 1clientDownload: 10 : file "whiskys.lair/zzz_all_rifles_v1.5.iwd" completed
SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = "213.179.252.246" LIMIT 1SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = "213.179.252.246" LIMIT 1Segmentation fault (core dumped)

---

Code:

---

Hitch warning: 3238 msec frame time
Resolving cod2master.activision.com
cod2master.activision.com resolved to 185.34.104.231:20710
Sending heartbeat to cod2master.activision.com
Resolving cod2master.activision.com
cod2master.activision.com resolved to 185.34.104.231:20700
Client 0 connecting with 600 challenge ping from 39.54.158.44:28960
Going from CS_FREE to CS_CONNECTED for  (num 0 guid 0)
Sending heartbeat to cod2master.activision.com
SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = "39.54.158.44" LIMIT 1SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = "39.54.158.44" LIMIT 1Client 1 connecting with 100 challenge ping from 109.94.125.169:28960
Going from CS_FREE to CS_CONNECTED for  (num 1 guid 0)
Rcon from 89.36.219.214:-24539:
say
SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = "109.94.125.169" LIMIT 1SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = "109.94.125.169" LIMIT 1Segmentation fault (core dumped)

---

Sorry for not adding new line in printf.

Got only dump from the second one:

Code:

---

warning: Could not load shared library symbols for ./libcod2_1_0.so.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Core was generated by `./cod2_lnxded +set net_port 28960 +set dedicated 2 +set fs_game whiskys.lair +e'.
Program terminated with signal 11, Segmentation fault.
#0  0xb716ab4b in mysql_options ()
  from /usr/lib/i386-linux-gnu/libmysqlclient.so.18
(gdb) bt
#0  0xb716ab4b in mysql_options ()
  from /usr/lib/i386-linux-gnu/libmysqlclient.so.18
#1  0xb773ccb1 in ?? ()
#2  0x00000000 in ?? ()

---

Any thoughts? :)

Looks like the same crash, segmentation fault 11 in some function using mysql_options(), so the code is dereferencing a NULL pointer e.g., which causes the segfault.

Probably still the easiest way to debug this:

Quote:

---

Originally Posted by kung foo man

Kinda filthy to debug a C API crash, to be able to analyze those crashes in my MySQL part I added debug statements in every C glue function, like:

https://github.com/kungfooman/libcod...mysql.cpp#L483

Code:

---

    #if DEBUG_MYSQL
    printf("gsc_mysql_num_fields(result=%d)\n", result);
    #endif

---

If I would want to tackle the problem, I would first give every function a debug printf, starting with the functions which call mysql_options(), since that was mentioned in the gdb backtrace.

---

I would printf the lines here: https://github.com/kungfooman/libcod...mysql.cpp#L290

Googling "mysql_options seg fault" shows some problems with mysql_options() too, but that stuff seems ancient. But could still be related to your libmysqlclient version maybe.

Code:

---

./cod2_lnxded_1_0: /usr/lib/i386-linux-gnu/libmysqlclient.so.18: no version information available (required by ./libcod2_1_0.so)

---

seems interesting...

Quote:

---

David, this is MySQL problem, they screwed something in their setup on Debian systems. There is lot of information on Internet about that, for example:

http://forum.directadmin.com/showthread.php?t=46807

try to follow their recommendations how to fix it.

You can just google " libmysqlclient.so.18: no version information available

---

Quote:

---

Originally Posted by kung foo man

Kinda filthy to debug a C API crash, to be able to analyze those crashes in my MySQL part I added debug statements in every C glue function, like:

https://github.com/kungfooman/libcod...mysql.cpp#L483

Code:

---

        #if DEBUG_MYSQL
        printf("gsc_mysql_num_fields(result=%d)\n", result);
        #endif

---

If I would want to tackle the problem, I would first give every function a debug printf, starting with the functions which call mysql_options(), since that was mentioned in the gdb backtrace.

---

I would like to adapt your idea, but I'm not sure I do this right way.

PHP Code:

---

SearchForBan()   
{   
    myIP = self getIP();  
    if(!isDefined(myIP)) 
        return; 
    printf("SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = \"%\" LIMIT 1\n", myIP);   
    maps\mp\gametypes\_mysql::asyncQuery("SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = \"" + myIP + "\" LIMIT 1", ::checkBan);   
    printf("SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = \"%\" LIMIT 1\n", myIP);
    printf("gsc_mysql_num_fields(result=%d)\n", checkBan);   
} 


---

Quote:

---

Originally Posted by IzNoGoD

Code:

---

./cod2_lnxded_1_0: /usr/lib/i386-linux-gnu/libmysqlclient.so.18: no version information available (required by ./libcod2_1_0.so)

---

seems interesting...

Quote:

---

David, this is MySQL problem, they screwed something in their setup on Debian systems. There is lot of information on Internet about that, for example:

http://forum.directadmin.com/showthread.php?t=46807

try to follow their recommendations how to fix it.

You can just google " libmysqlclient.so.18: no version information available

---

---

Yeah I googled it and found that AskUbuntu. I also run Ubuntu 12.04, but I have the ability to upgrade it to 14.04. Shall I do this?

Quote:

---

Silly question:

If I run 'do-release-upgrade' to upgrade the ubuntu, I won't loose any private datas (/home/ or any other directory (/var/ is also important to me :) ) nor mysql database). ?

---

Sorry for being too lazy to google it. Already did.

EDIT:

Seems that the mysql problem is only related to 5.5 verison and I have (5.5.47-0ubuntu0.12.04.1) :|

what does apt-get dist-upgrade do for your mysql version?

Normally you should be able to just copy the working libmysqlclient.so over next to your cod2_lnxded binary and then "include" it with LD_PRELOAD_PATH=. (IIRC, did this stuff years ago)

This one is from 2009, might still do it's job great on your Ubuntu 12.04: http://killtube.org/downloads/libcod/libmysqlclient.so (or take the one from your working distribution)

OR:

Rename the default one to /usr/lib/i386-linux-gnu/libmysqlclient.so.18_backup and upload the working one from your working distribution (hacky hacky..)

The command "ldd cod2_lnxded" is pretty helpful to see which .so would be loaded.

Also try adding LD_DEBUG=libs for more debugging: http://www.bnikolic.co.uk/blog/linux-ld-debug.html


About the printf debug... I meant to add printf's in the libcod source code and recompiling it, but seems like we already know that it is the MySQL lib version, so no need for that anymore.

I'm still fighting with this problem :P.

What I've done so far:

1). Worked with Iznogod and his libcod fixes (sadly, it didn't fix the problem :|)
2). Upgraded Ubuntu to 14.04
3). Updated linux-libc-dev i386 to version 3.13.0-85.129 (Dunno if it's even related but it was outdated and I was updating everything I could)
4). Added LD_DEBUG=all to start script.
5). It seems that after the dist upgrade segfaults aren't saving core dumps. Tried ulimit -c unlimited, but it resets later (Checking it with ulimit -a). Any other way to save core dumps?
6). Before last segfault server skipped query (maybe it is related to the problem?)

Code:

---

Client 47 connecting with 250 challenge ping from 37.238.162.56:-31071
Going from CS_FREE to CS_CONNECTED for  (num 7 guid 0)
clientDownload: 7 : begining "whiskys.lair/zzz_modz.iwd"
clientDownload: 7 : file "whiskys.lair/zzz_modz.iwd" completed
clientDownload: 7 : begining "whiskys.lair/zzz_all_rifles_v1.5.iwd"
clientDownload: 7 : file "whiskys.lair/zzz_all_rifles_v1.5.iwd" completed
SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = "37.238.162.56" LIMIT 1
SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = "37.238.162.56" LIMIT 1
Sending heartbeat to cod2master.activision.com
3: EXE_DISCONNECTED
^9(GAME_AXIS)I<3evil.Server 445^7: QUICKMESSAGE_MOVE_IN
Client 48 connecting with 150 challenge ping from 78.38.19.196:10039
Going from CS_FREE to CS_CONNECTED for  (num 3 guid 0)
4: EXE_DISCONNECTED
Client 49 connecting with 250 challenge ping from 41.225.131.123:28960
Going from CS_FREE to CS_CONNECTED for  (num 16 guid 0)
13: EXE_DISCONNECTED
SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = "41.225.131.123" LIMIT 1
SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = "41.225.131.123" LIMIT 1
Rcon from 89.36.219.214:-10854:
say
Segmentation fault (core dumped)

---

PHP Code:

---

SearchForBan()
{
    myIP = self getIP();
    if(!isDefined(myIP))
        return;
    printf("SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = \"%\" LIMIT 1\n", myIP);  
    maps\mp\gametypes\_mysql::asyncQuery("SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = \"" + myIP + "\" LIMIT 1", ::checkBan);
    printf("SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = \"%\" LIMIT 1\n", myIP);    
} 


---

Seems like myIP wasn't defined, from now I've added printf before return.

Btw. maybe check if isDefined(self) before getting IP?

Attaching logfile with LD_DEBUG=all https://www.sendspace.com/file/kq3tie

Thanks again!

Let me drop the big UPDATE here:

1). I am retarded.
2). I've been backtracing the old core dump from wrong mysql initiating (which was later resolved).
3). New backtracing look like this:

Code:

---

#0  0x080e983d in ?? ()
(gdb) bt
#0  0x080e983d in ?? ()
#1  0x0808f697 in ?? ()
#2  0x0808fa6f in ?? ()
#3  0x080943e1 in ?? ()
#4  0x08061779 in ?? ()
#5  0x08062451 in ?? ()
#6  0x0806251d in ?? ()
#7  0x080d2b22 in ?? ()
#8  0xb738aa83 in __libc_start_main (main=0x80d2990, argc=12, argv=0xbfd22604,
    init=0x8049b90, fini=0x8139b50, rtld_fini=0xb7722180 <_dl_fini>,
    stack_end=0xbfd225fc) at libc-start.c:287
#9  0x0804a4d1 in ?? ()

---

4). Thanks to Kung's lessons I've managed to check the code behind those addresess:
Behind #1 0x0808f697 in ?? () there is:

PHP Code:

---

int __cdecl sub_808F510(int a1, int a2, int a3)
{
  int result; // eax@4
  char v4; // [sp+4h] [bp-3D4h]@0
  int v5; // [sp+14h] [bp-3C4h]@9
  char *v6; // [sp+18h] [bp-3C0h]@9
  char *v7; // [sp+1Ch] [bp-3BCh]@11
  char v8[896]; // [sp+20h] [bp-3B8h]@11
  char s; // [sp+3A0h] [bp-38h]@9
  int v10; // [sp+3C4h] [bp-14h]@5
  int v11; // [sp+3C8h] [bp-10h]@9
  int i; // [sp+3CCh] [bp-Ch]@9
  char v13[8]; // [sp+3D0h] [bp-8h]@25

  if ( a3 )
    *(_DWORD *)(a1 + 134408) = *(_DWORD *)(a1 + 133144);
  else
    *(_DWORD *)(a1 + 134408) = -1;
  result = *(_DWORD *)(a1 + 133132) - *(_DWORD *)(a1 + 133136);
  if ( result <= 127 )
  {
    v10 = sub_8067EE8(a2);
    if ( v10 > 0 )
    {
      if ( v10 <= 32 )
      {
        v11 = dword_842BC8C;
        v11 = *(_DWORD *)(a1 + 133144) ^ dword_842BC8C;
        v11 ^= sub_80AA70C(a1 + 1032 * (*(_DWORD *)(a1 + 133136) & 0x7F) + 1036, 32);
        v5 = sub_808FE02(-1653759219 * ((a1 - (signed int)dword_841FB0C) >> 2));
        sub_80684E6(v5, &s);
        v6 = &s;
        for ( i = 0; i < v10; ++i )
        {
          v7 = &v8[28 * i];
          sub_8068ADE(a2, v11, v6, v7);
          if ( !(unsigned __int8)sub_80E97F0(v5, (unsigned __int8)v7[8]) )
            v7[8] = *(_BYTE *)(v5 + 212);
          if ( !(unsigned __int8)sub_80E97F0(v5, (unsigned __int8)v7[9]) )
            v7[9] = *(_BYTE *)(v5 + 208);
          v6 = v7;
        }
        *(_DWORD *)(a1 + 9924 * (*(_DWORD *)(a1 + 133144) & 0x1F) + 144352) = dword_841FB04;
        if ( *(_DWORD *)a1 == 3 )
          sub_808E1F0(a1, v8);
        if ( !*(_BYTE *)(dword_848B200 + 8) || *(_DWORD *)(a1 + 452016) )
        {
          if ( *(_DWORD *)a1 == 4 )
          {
            for ( i = 0; ; ++i )
            {
              result = i;
              if ( i >= v10 )
                break;
              if ( *(_DWORD *)&v13[28 * i - 944] <= *(_DWORD *)&v13[28 * v10 - 972]
                && *(_DWORD *)&v13[28 * i - 944] > *(_DWORD *)(a1 + 133156) )
              {
                sub_808F488(a1, &v8[28 * i]);
              }
            }
          }
          else
          {
            result = a1;
            *(_DWORD *)(a1 + 134408) = -1;
          }
        }
        else
        {
          result = sub_808DC8C(a1, "EXE_CANNOTVALIDATEPURECLIENT");
        }
      }
      else
      {
        result = sub_8060B2C("cmdCount > MAX_PACKET_USERCMDS\n", v4);
      }
    }
    else
    {
      result = sub_8060B2C("cmdCount < 1\n", v4);
    }
  }
  return result;
} 


---

5). Today on the server which partly uses my scripts I've noticed this:
http://i.imgur.com/HVVa5hn.jpg
It was spamming the hell out of the console.
6). thOuMta (server's admin) did not see it (only me).
7). To stop this spam I've used the cl_maxpackets "30" command. (this server had set 20 at default).
8). Kung foo man has pointed out the problem with "cmdCount > MAX_PACKET_USERCMDS".

Not sure what to do about it, Should I patch libcod with this? https://killtube.org/showthread.php?...tch-must-apply or just parse cl_maxpackets?
Or.. maybe as it was said in this thread https://killtube.org/showthread.php?...ACKET_USERCMDS I have "wait without ending on disconnect"
Could my self SearchForBan(); be treated as a wait and I should move it somewhere?

PHP Code:

---

SearchForBan()
{
    if(!isDefined(self))
    {
        printf("\n Self wasn't defined :(");
        return;
    }
    myIP = self getIP();
    if(!isDefined(myIP))
    {
        printf("\n myIP wasnt defined! \n");
        return;
    }
    printf("SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = \"%\" LIMIT 1\n", myIP);  
    maps\mp\gametypes\_mysql::asyncQuery("SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = \"" + myIP + "\" LIMIT 1", ::checkBan);
    printf("SELECT `ip`, `name`, `reason`, `whobanned` FROM `bans` WHERE `ip` = \"%\" LIMIT 1\n", myIP);    
}

checkBan(rows, args)
{
    
   if(isDefined(rows) && isDefined(rows[0]) && isDefined(rows[0]["ip"]) && isDefined(self))
   {
           kickmsg = rows[0]["name"] + "^7, You're banned by " + rows[0]["whobanned"] + "^7, because of reason: " + rows[0]["reason"] + ". Check www.whiskaskitler.xyz to get unbanned :) " + ". You will need your IP: " + rows[0]["ip"];
          kick2(self getEntityNumber(), kickmsg);
   }
}

Callback_PlayerConnect()
{
    thread dummy();
    
    self.statusicon = "hud_status_connecting";
    self waittill("begin");
    self.statusicon = "";
    level notify("connected", self);
    
    if(!level.splitscreen)
        iprintln(&"MP_CONNECTED", self);
    lpselfnum = self getEntityNumber();
    lpGuid = self getGuid();
    logPrint("J;" + lpGuid + ";" + lpselfnum + ";" + self.name + "\n");

    if(game["state"] == "intermission")
    {
        spawnIntermission();
        return;
    } 
    level endon("intermission");
    
    self SearchForBan();     

    if(level.splitscreen)
        scriptMainMenu = game["menu_ingame_spectator"];
    else
        scriptMainMenu = game["menu_ingame"];

    if(isDefined(self.pers["team"]) && self.pers["team"] != "spectator")
    {
        self setClientCvar("ui_allow_weaponchange", "1");

        if(self.pers["team"] == "allies")
            self.sessionteam = "allies";
        else
            self.sessionteam = "axis";

        if(isDefined(self.pers["weapon"]))
            spawnPlayer();
        else
        {
            spawnSpectator();

            if(self.pers["team"] == "allies")
            {
                self openMenu(game["menu_weapon_allies"]);
                scriptMainMenu = game["menu_weapon_allies"];
            }
            else
            {
                self openMenu(game["menu_weapon_axis"]);
                scriptMainMenu = game["menu_weapon_axis"];
            }
        }
    }
    else
    {
        self setClientCvar("ui_allow_weaponchange", "0");

        if(!isDefined(self.pers["skipserverinfo"]))
            self openMenu(game["menu_team"]);

        self.pers["team"] = "spectator";
        self.sessionteam = "spectator";

        spawnSpectator();
    }

    self setClientCvar("g_scriptMainMenu", scriptMainMenu);
} 


---

Sorry for the long post. I've wanted to make it with as many details as (I thought) it needs.

Thank you.

Edit: used cl_maxpackets "20" on my server and also got spammed.

The rate limit patch is not related to crashes. It is for preventing people from generating too much traffic on your server.

I never found why the server crashes when you are seeing a few cmdCount > MAX_PACKET_USERCMDS in your log.
How often does this crash happen and are you able to replicate the crash? Replicating the crash will make it easier to test and fix the issue.

I don't think cmdCount > MAX_PACKET_USERCMDS (32) is the cause of the crash.
https://github.com/id-Software/Quake...er/sv_client.c

#0 0x080e983d traces back to:

PHP Code:

---

int __cdecl sub_80E97F0(int a1, int a2)
{
  int v3; // [sp+Ch] [bp-Ch]@5
  char v4; // [sp+17h] [bp-1h]@1

  v4 = 1;
  if ( !(unsigned __int8)sub_80E9758(a2) )
    v4 = 0;
  if ( !(unsigned __int8)sub_80D9E84(a1 + 1348, a2) )
    v4 = 0;
  v3 = sub_80E9270(a2); // weapon = getweapon(index)
  if ( !*(_DWORD *)(v3 + 132) // = 0x080e983d = weapon offhandClass
    && *(_BYTE *)(a1 + 1365) != a2
    && *(_BYTE *)(a1 + 1366) != a2
    && *(_DWORD *)(v3 + 876) != a2 )
    v4 = 0;
  return (unsigned __int8)v4;
} 


---

PHP Code:

---

v7 = &v8[28 * i];
sub_8068ADE(a2, v11, v6, &v8[28 * i]);
if ( !(unsigned __int8)sub_80E97F0(v5, (unsigned __int8)v7[8]) ) // probably player entity, weapon index 


---

It could that for some reason the weapon index is out of range.
One way to found out is to hook the sub_80E9270 call and print a message when the index is invalid.

Quote:

---

Originally Posted by Mitch

How often does this crash happen and are you able to replicate the crash? Replicating the crash will make it easier to test and fix the issue.

---

Could be 10 crashes per day. Usually one by one, then peace for next ~14 hours. I didn't tried to replicate it :(.

Quote:

---

Originally Posted by Mitch

#0 0x080e983d traces back to:

PHP Code:

---

int __cdecl sub_80E97F0(int a1, int a2)
{
  int v3; // [sp+Ch] [bp-Ch]@5
  char v4; // [sp+17h] [bp-1h]@1

  v4 = 1;
  if ( !(unsigned __int8)sub_80E9758(a2) )
    v4 = 0;
  if ( !(unsigned __int8)sub_80D9E84(a1 + 1348, a2) )
    v4 = 0;
  v3 = sub_80E9270(a2); // weapon = getweapon(index)
  if ( !*(_DWORD *)(v3 + 132) // = 0x080e983d = weapon offhandClass
    && *(_BYTE *)(a1 + 1365) != a2
    && *(_BYTE *)(a1 + 1366) != a2
    && *(_DWORD *)(v3 + 876) != a2 )
    v4 = 0;
  return (unsigned __int8)v4;
} 


---

PHP Code:

---

v7 = &v8[28 * i];
sub_8068ADE(a2, v11, v6, &v8[28 * i]);
if ( !(unsigned __int8)sub_80E97F0(v5, (unsigned __int8)v7[8]) ) // probably player entity, weapon index 


---

---

Yeah, Kung foo man suggested to hook up the address into libcod:

PHP Code:

---

#if COD_VERSION == COD2_1_0
            if (0)
                cracking_hook_function(0x08092D5C, (int)SV_AddServerCommand);
            if (0)
                cracking_hook_function(0x0809301C, (int)SV_SendServerCommand);

            cracking_hook_call(0x0808F134, (int)hook_ClientUserinfoChanged);
            cracking_hook_call(0x0807059F, (int)Scr_GetCustomFunction);
            cracking_hook_call(0x080707C3, (int)Scr_GetCustomMethod);
            cracking_hook_call(0x08098CD0, (int)custom_SV_WriteDownloadToClient);
            cracking_hook_call(0x080DFF66, (int)hook_player_setmovespeed);
            cracking_hook_call(0x080F50AB, (int)hook_player_g_speed);
            cracking_hook_call(0x080E9524, (int)hook_findWeaponIndex);
            cracking_hook_call(0x080E983D, (int)hook_findProblem); // This address 


---

and add printf with parameters in gsc_player.cpp

PHP Code:

---

int hook_findProblem(int a1, int a2) { 
    printf("findProblem: %d %d\n", a1, a2);
    return 1; 
} 


---

Also added function to gsc_player.hpp

PHP Code:

---

int hook_findProblem(int a1, int a2); 


---

We hoped that it fill print arguments before server crash. But at the next ~10 segfaults there was no output from this address. So I've started to check out previous addresses and as I found addresses with MAX_PACKET kung suggested to check out previous threads on the forum.

Quote:

---

Originally Posted by Mitch

It could that for some reason the weapon index is out of range.
One way to found out is to hook the sub_80E9270 call and print a message when the index is invalid.

---

Will do this, but yesterday I've implemented your libcod version, parsed cl_maxpackets on PlayersConnects and moved SearchForBan(); at the end of the Callback_PlayerConnect(). For now I didn't have any segfaults which is record for my server. If segmentation fault will came back I let you know with provided printf of sub_80E9270. Thanks for your time!

Whisky