7 replies to this topic

[Serg]

Hello.
I don't use transactions in my program. I use algorithm shown below:

public boolean setValue(int a_new_value)
{
this.exclusiveLock();
if(this.value != a_new_value)
{
  this.value = a_new_value;
  this.modify();
}
...........
if(this.isModified())
  this.store();
this.unlock();
}

This code invokes in different concurrent threads. Perst storage works with INFINITE_PAGE_POOL, and naturally it use StrongHashTable. So, when I invoke IPersistent.store(), code in StrongHashTable works:

public synchronized void setDirty(Object obj) {
if (nModified < MODIFIED_BUFFER_SIZE) {
modified[nModified] = obj;
}
nModified += 1;
}

The variable "nModified" will reset never in my code because I don't use Storage.commit(), and "modified[]" will full always.
After my program has worked 6 months the variable "nModified" overflowed.

How I must change my code to prevent this problem?

PS I don't use Storage.commit() because I think that .store() more quick. Or I mistake?

[perstmco]

I don't really understand your use case. If you are not committing any transaction, then why do you call the store method at all?
What is the need to store something in the storage, if in the case of a failure all your changes will be lost (even if they are stored in the database - because all uncommitted transactions are rolled back). Right now, the close() method implicitly performs a commit. So you can just remove all invocations of store() and let the Storage.close method save all modified objects.

In any case, we will change the type of nModified in the object cache from int to long, to avoid overflow (because it is possible that a transaction updates more than 2^32 objects).

[Serg]

Thanks for your answer.

I understand that I need to commit my changes. But in my case the intensity of storage change is very high, and this changes performs in different threads. So I can't invoke commit to every change, because it decrease performance a lot. And I want to get information about size of modified objects. It will be very good if method modify() return boolean (for example). This boolean tell me that "modified[]" is full and I perform commit. I want to get indication about "modified[]".
And how I can see in StrongHashTable class, increment nModified variable more than MODIFIED_BUFFER_SIZE is not necessary, because it used as flag for selection flush method. The value more than MODIFIED_BUFFER_SIZE is not used.

[perstmco]

Your application has not committed changes for more than 6 months, which caused counter overflow. It can be very inefficient to perform a commit after each update. But you can, for example, perform a commit after each 100-th update. Or use time criteria and perform a commit after each 10 seconds (or ten minutes - whatever you want).

Changing profile of the Persistent .modify method to make it to return some boolean value is not possible. We must preserve backward compatibility and some users redefine this method in their classes.

Also, returning true if the number of modified objects in strong hash exceeds some strong hash specific threshold MODIFIED_BUFFER_SIZE seems to be very confusing behaviour - because modify should work with all cache managers and its behaviour should not depend on the cache manager used.

StrongHashTable class behavior cannot be changed to not to increment the nModified variable when it exceeds MODIFIED_BUFFER_SIZE because it is used in flush and is expected to be incremented during flush (when recursive save of some object cause saving of some other objects).

What be done is to just replace int type for nModified with long. It will be included in next Perst release.

[Serg]

Thanks for replies.

[Serg]

So, I use commit every 1000 times. But I don't understand I must use .store() or I can use only .modify() ? What .store() do?

[perstmco]

>> But I don't understand I must use .store() or I can use only .modify() ? What .store() do?

modify() just marks an object as having been modified. It will be saved during transaction commit. store() saves the object immediately.

Using modify() may be more efficient if the same object is updated multiple times within a transaction. But, on the other hand, modified objects are pinned in memory, so if there are long running transaction and too many modified objects, it can cause memory overflow.

[Serg]

Thanks.