Archived
This forum has been archived. Please start a new discussion on GitHub.
Icestorm<->Berkeley DB error
Hello,
i have a function that reads several datasets from an Oracle DB. For each of these datasets I want to create an Icestorm-topic.
After I created approx. 350 topics I get the following error message:
[ icebox-IceStorm: Berkeley DB: DBEnv "db:" Lock table is out of locker entries ]
icebox-IceStorm: warning: dispatch exception: SharedDb.cpp: 180: Freeze::DatabaseException
DB::open: Not enough space
identity: IceStorm/TopicManager
facet:
operation: create
Are there any configuration values that I missed to set?
Thanks in advance.
i have a function that reads several datasets from an Oracle DB. For each of these datasets I want to create an Icestorm-topic.
After I created approx. 350 topics I get the following error message:
[ icebox-IceStorm: Berkeley DB: DBEnv "db:" Lock table is out of locker entries ]
icebox-IceStorm: warning: dispatch exception: SharedDb.cpp: 180: Freeze::DatabaseException
DB::open: Not enough space
identity: IceStorm/TopicManager
facet:
operation: create
Are there any configuration values that I missed to set?
Thanks in advance.
0
Comments
-
Hello,
This is a Berkeley DB configuration issue. By creating lots of topics, you've reached the maximum number of Berkeley DB lockers configured by default, 1000. See http://www.sleepycat.com/docs/api_cxx/env_set_lk_max_lockers.html and http://www.sleepycat.com/docs/ref/lock/max.html.
The easiest solution is to create a DB_CONFIG file in your db directory and specify set_lk_max_lockers <some larger number>, for example:
set_lk_max_lockers 10000
Note that the Berkeley DB documentation at http://www.sleepycat.com/docs corresponds to the DB 4.2.52 release (from Nov 25). Ice 1.2 uses 4.1.25 which is slightly different, for example it does not have the various get_ functions to retrieve configuration settings.
Cheers,
Bernard0 -
Originally posted by bernard
Hello,
This is a Berkeley DB configuration issue. By creating lots of topics, you've reached the maximum number of Berkeley DB lockers configured by default, 1000. See http://www.sleepycat.com/docs/api_cxx/env_set_lk_max_lockers.html and http://www.sleepycat.com/docs/ref/lock/max.html.
The easiest solution is to create a DB_CONFIG file in your db directory and specify set_lk_max_lockers <some larger number>, for example:
set_lk_max_lockers 10000
What's the relationship between DB lockers and topics? Does each topic hold a cursor and DB handle, or more? Is 3-to-1 a decent scaling ratio?
I've been considering a design that relies on a lot (like, tens of thousands) of IceStorm topics, so I'm interested in any experiences people have had scaling IceStorm to that level.
Mike0 -
Hello,
I looked into BerkeleyDB-docs and created a config file (DB_CONFIG) with the following entries:
set_lk_max_lockers 10000
set_lk_max_locks 10000
set_lk_max_objects 10000
On my windows machine I can only create 817 topics. There is no difference if the value of set_lk_max_lockers is set to 10000 or 100000.
Then I've tested my application on a Solaris machine. After only 116 topics the following error occurs:
[ icebox-IceStorm: Berkeley DB: DbEnv "db": DB_TXN->abort: Log undo failed for LSN: 1 1946136: DB_NOTFOUND: No matching key/data pair found ]
[ icebox-IceStorm: Berkeley DB: DbEnv "db": PANIC: DB_NOTFOUND: No matching key/data pair found ]
[ icebox-IceStorm: Berkeley DB: DbEnv "db": fatal region error detected; run recovery ]
[ icebox-IceStorm: Berkeley DB: DbEnv "db": fatal region error detected; run recovery ]
[ icebox-IceStorm: Berkeley DB: DbEnv "db": PANIC: DB_RUNRECOVERY: Fatal error, run database recovery ]
[ icebox-IceStorm: Berkeley DB: DbEnv "db": fatal region error detected; run recovery ]
icebox-IceStorm: warning: dispatch exception: SharedDb.cpp:180: Freeze::DatabaseException:
Db::open: DB_RUNRECOVERY: Fatal error, run database recovery
identity: IceStorm/TopicManager
facet:
operation: create
An additional problem is the performance of the topic creation under both systems. For example, the creation of approx. 280 topics on the windows machine takes one minute.
Thanks in advance0 -
Hello,
I've just created 10,000 topics on my Windows XP laptop using the following DB_CONFIG:
set_lk_max_lockers 40000
set_lk_max_locks 40000
set_lk_max_objects 40000
set_lg_regionmax 2400000
set_cachesize 0 104857600 1
set_lg_regionmax is necessary because of the large number of files, and you need a larger set_cachesize most likely as a work-around for bug #6681 (see http://www.sleepycat.com/update/4.2.52/if.4.2.52.html).
I haven't tried on Solaris. Maybe you reached a file descriptors limit?
Anyway, creating lots of topics in IceStorm should not be a Berkeley DB scalability exercise. It also takes too long. IceStorm should not create a Freeze Map (and hence Berkeley DB database) per topic ... everything could be combined in one big Map. We'll try to fix this for the next Ice release.
Cheers,
Bernard0 -
I've been working on pulling out the TopicManager/Topic storage stuff into a separate StormStorage class for the past day or two, with the goal to be able to provide both a Freeze-backed persistent set of topics/links, as well as a transient in-memory-only one. This probably isn't as interesting if the maps get condensed into one, but I'll post a patch anyway when I get things to stop crashing.0
-
Hello,
IceStorm in Ice 1.3 was updated to use a single database for all topics. As a result topic creation is a lot faster now. And you can create a very large number of topics without any special Berkeley DB configuration.
However, if you're creating several thousand topics and want to speed up their creation, I recommend you create a DB_CONFIG file in your Berkeley DB directory (typically .../db/) with the following directive:
set_flags DB_TXN_NOSYNC
See http://www.sleepycat.com/docs/api_cxx/env_set_flags.html for a description of DB_TXN_NOSYNC.
Cheers,
Bernard0 -
Hello,
I created a DB_CONFIG file and the creation of many topics is much faster. Thanks a lot for this. Anyway, are there any possibilities to improve the performance a little bit more?
Thanks in advance.0 -
If you don't care about topic/links persistence, and really care about topic-creation-speed, removing Freeze altogether would make sense. Did you try vukicevic's patch?
Cheers,
Bernard0