Forum ‹ C# ‹ Server Lag Spikes

[FulaFisken]

Hi there.

Recently I have had quite a few lag spikes occuring on the server in our game loop.

Code: Select all

public void Update()
{
    DateTime newTime = DateTime.Now;
    frameTime = (newTime - currentTime).TotalMilliseconds;
    currentTime = newTime;

    if (frameTime > 1000)
    {
        PlayerIO.ErrorLog.WriteError("Server lag spike, > 1000 ms");
        frameTime = 1000;
    }

    accumulator += frameTime;

    while (accumulator >= PERIOD)
    {
        shipManager.Update();
        bodyManager.Update();
        projectileManager.Update();
        dropManager.Update();
        spawnManager.Update();
        turretManager.Update();

        accumulator -= PERIOD;
    }

}


Now, this did not happen at all before (+2 weeks ago) but occurs at least 2-3 times each day now. What kind of stuff can cause this? Dead-locks, exceptions or just something that takes really long to execute? Is this common or something really dangerous for our game? The player will see this as a lag spike, but then its corrected and it doesn't snowball into something bigger as far as we know yet.

[Oliver]

Hello,

Is updated called on a Timer?

- Oliver

[FulaFisken]

Hello,

Is updated called on a Timer?

- Oliver

Yes.

Edit:

Code: Select all

public int PERIOD = 33;

// physics timer
AddTimer(delegate
{
    Update();
}, PERIOD);

[Oliver]

It can happen if the server the room is running is overloaded or has code that misbehaves, or suddenly has to process a large amount of events at the same time, then events will (internally) be served on a first-come-first-serve basis.

We monitor and distribute load across the servers in an effort to avoid one game having an impact on another, but it does happen once in a while; it's one of the downsides of shared hardware. We do offer dedicated servers though, which i'd be happe to discuss with you.

Best,
Oliver

[FulaFisken]

It can happen if the server the room is running is overloaded or has code that misbehaves, or suddenly has to process a large amount of events at the same time, then events will (internally) be served on a first-come-first-serve basis.

We monitor and distribute load across the servers in an effort to avoid one game having an impact on another, but it does happen once in a while; it's one of the downsides of shared hardware. We do offer dedicated servers though, which i'd be happe to discuss with you.

Best,
Oliver

Ok. Thanks!

Yes, dedicated servers would be very useful when our game is up and running. We are about to release a beta soon (http://www.wowwiki.com/Soon), so stay tuned

[FulaFisken]

It can happen if the server the room is running is overloaded or has code that misbehaves, or suddenly has to process a large amount of events at the same time, then events will (internally) be served on a first-come-first-serve basis.

A naive question. Would it be horrible for performance to have our Update method completely synchronized with a lock? Thus preventing the next update event to wait before the other is finished.

Does this mean theoretically that every unit in the game can get the wrong value (overwritten by the overlapping thread) on all properties that should be updated after another, ie. synchronized, such as position, speed, etc.

[Oliver]

I think it would be okay to have the Update() method inside a lock. In the normal (99.999%) case, you'll never have overlapping executions of it, so other threads won't have to wait to get the lock, so it shouldn't slow anything down except for the extra time to get and release the lock.

[FulaFisken]

I think it would be okay to have the Update() method inside a lock. In the normal (99.999%) case, you'll never have overlapping executions of it, so other threads won't have to wait to get the lock, so it shouldn't slow anything down except for the extra time to get and release the lock.

Interesting, cause we have noticed something that might be caused by this. We will try to lock it and see if we get any different result.

[FulaFisken]

Adding a lock to our physics update did wonders for our game. No more odd anomalities where units get wierd values. I will keep posting if something unforseen occurs.

[FulaFisken]

Adding a lock to our physics update did wonders for our game. No more odd anomalities where units get wierd values. I will keep posting if something

Hi again! Thanks for all the replies.

After testing our game for a while we still get a lot of hickups on the server, we test played a bit for 3-4 hours and got 89 hickups. We have very few CPU aborts (1 this month), I don't know if these are related in any kind of way.

The lock inside the update method didn't increase the hickup count, but at least it got rid of the issues the hickups caused with overlapping threads.

Querstion
Now, my guess is that we might be overloading the server, or having a dead-lock. But its hard to identify the reason for the hickups (>1000ms), our game might just be to heavy for the server or we have some real issues in our code. But if dead-lock is the case, doesn't it show as a CPU abort?

We will continue to monitor this.

//Stefan