Forum ‹ BigDB ‹ BigDB LoadRange() limit *changed without warning?!*

[wildbunny]

Hi guys,

Just looked in my log today to find that I've got an error:

"Limit can not be larger than 1000, argument out of range: 10000"

Now, this *was fine* before, so something must have changed. Why weren't we notified?

Also, how am I supposed to query with no limit? I tried 0 and -1 in limit and both returned 0 objects! What gives?

Ta, Paul.

[FulaFisken]

Hi guys,

Just looked in my log today to find that I've got an error:

"Limit can not be larger than 1000, argument out of range: 10000"

Now, this *was fine* before, so something must have changed. Why weren't we notified?

Also, how am I supposed to query with no limit? I tried 0 and -1 in limit and both returned 0 objects! What gives?

Ta, Paul.

Hasn't there always been a limit of 1000? Also, the limit is probably there for a purpose, I would think it is intended that you are required to use a limit for performance reasons. Why would you need to query for more than the limit in one load, seem kind of unnecessary?

[wildbunny]

Hasn't there always been a limit of 1000? Also, the limit is probably there for a purpose, I would think it is intended that you are required to use a limit for performance reasons. Why would you need to query for more than the limit in one load, seem kind of unnecessary?

If there was it wasn't enforced, because I've not changed my code and this is the first time I'm seeing this error.

I wanted to query limitless but the documentation is lacking any indication on how to do that, so 10000 seemed as good as any large value.

[Oliver]

Hello,

All BigDB operations are supposed limited to loading of 1000 objects. If you try to load objects by key, you'll get the same message.

However, we discovered a bug that ment that this behavior was not enforced in in the LoadRange method. The bug was fixed, which caused the behavior you're seeing.

We're very sorry for the inconvenience.

You can still load all your objects if you want, you just have to do it over multiple loads moving over the index.

Best,
Oliver

[wildbunny]

We're very sorry for the inconvenience.

You can still load all your objects if you want, you just have to do it over multiple loads moving over the index.

Best,
Oliver

How can I LoadRange() with no limit?

[Oliver]

You have to do it in increments over the values. So, multiple loadrange, depending on how much data.

Something like this: (this is very fast pseudo code, so it' probably not very sane).

while(true){
var start = null;
values = LoadRange(from:start to:null limit:1000)
if( values .length != 1000 ){ // end
break;
}else{
start = max(values);
}
}


In general, I'd ask why you'd want to load more than 1000 entries in one go? It's a lot of data to be pushing back and forward; and (depending on whether your data is unbounded) you'll get worse and worse performance as your data grows, since it has to load more and more data each time.

Best,
Oliver