Jump to content
McObject Forums
Sign in to follow this  
Mikkel

Problem with First Perst App, where is my mistake?

Recommended Posts

Hello,

first I have to say that is my first App using perst. Normally the performance wasn't important for me, or rather sqlite was fast enough.

Actually I have some Problems understanding to work with perst. I read the tutorial and I think I made everything like it is explained there.

I try to save objects of MyData, is my way to do this correct?

Do I have to give own Keys to every object? Is there no way to generate automatic keys?


I tested my App, problem is, when I add 5 objects, my app counts 5 saved objects. If I then add 2 more objects of MyData, my App counts 2 saved objects. Why? Where is my mistake? I will Append my whole source code, beeing in hope someone here can help me.

thanks for reading and trying to help
Mikkel

My Code( sorry, I Dont't find the button for appending the files):

I initialize the database in method perstinit, Storing objects get made in perstInsertFakeData.

package de.gerding.perstperformance;

import java.util.ArrayList;
import java.util.Iterator;

import org.garret.perst.Index;
import org.garret.perst.Key;
import org.garret.perst.Persistent;
import org.garret.perst.Storage;
import org.garret.perst.StorageFactory;
import android.app.Activity;
import android.os.Bundle;
import android.view.View;
import android.widget.ArrayAdapter;
import android.widget.EditText;
import android.widget.GridView;
import android.widget.Toast;

public class PerstPerformanceActivity extends Activity
{
	private long time = 0;
	private GridView gridview_times;
	private EditText anzahl_datensaetze;
	private EditText anzahl_datenbytes;
	private ArrayList<String> ergebniszeiten = new ArrayList<String>() ;// = new String[] {"", ""}
	private ArrayAdapter<String> gridviewArrayAdapter;
	private String databaseName = "perstdb";
	final static int pagePoolSize = 2*1024*1024;
	private Storage db;
        private MyRootClass root;
    
    @Override
    protected void onCreate(Bundle savedInstanceState) 
    {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.perst_performance_activity_layout);
        gridview_times = (GridView) findViewById(R.id.gridView_times);
        anzahl_datensaetze = (EditText) findViewById (R.id.editText_anzahl_datensaetze);
        anzahl_datenbytes = (EditText) findViewById (R.id.editText_anzahl_datenbytes);
        gridviewArrayAdapter = new ArrayAdapter<String>(this, android.R.layout.simple_list_item_1, ergebniszeiten);
 		gridview_times.setAdapter(gridviewArrayAdapter);	
 		ergebniszeiten.add("time in ms");
        ergebniszeiten.add("Datensätze, Datenbytes");
        gridviewArrayAdapter.notifyDataSetChanged(); 
        perstinit();
    }
    public void perstinit()
    {
    	String Filepath = this.getFilesDir().getAbsolutePath();
		// Get instance of Perst storage
        db = StorageFactory.getInstance().createStorage();
        // Open the database with given database name and spacified page pool (database cache) size
        db.open(Filepath+databaseName, pagePoolSize);
        // There is one root object in the database. 
        root = (MyRootClass)db.getRoot();
        if (root == null) 
        {  
        	Toast.makeText(getApplicationContext(), "root Object is null", Toast.LENGTH_LONG).show();
        	// if root object was not specified, then storage is not yet initialized
            // Perform initialization:
            root = new MyRootClass(db);// ... create root object 
            // and register new root object
            db.setRoot(root);
        }        
    }
    public void saveFakeData(View v)
    {
    	int anzahl = Integer.valueOf(anzahl_datensaetze.getText().toString());
    	int byteanzahl = Integer.valueOf(anzahl_datenbytes.getText().toString());
    	ArrayList<MyData> fakedata = new ArrayList<MyData>();
    	fakedata= generateFakeData(anzahl, byteanzahl);
    	long minutestart = System.currentTimeMillis();
    	long minutestop = 0;
    	do
    	{
    		minutestop = System.currentTimeMillis();
    		time = perstInsertFakeData(fakedata);
        	ergebniszeiten.add(""+time);
        	ergebniszeiten.add(""+ anzahl+ ", " + byteanzahl);
        	gridviewArrayAdapter.notifyDataSetChanged();
    	}while((minutestop-minutestart)<0);
    	//}while((minutestop-minutestart)<60000);
    }

    private long perstInsertFakeData(ArrayList<MyData> dataarray) 
    {
    	long start = System.currentTimeMillis();
        // Create new object instance
        for(int i = 0; i < dataarray.size(); i++)
        {
        	MyData obj = dataarray.get(i);
            // ... and insert it in the corresponding indexes
            root.intKeyIndex.put(obj); // add object to index on intKey field
            root.strKeyIndex.put(obj); // add object to index in strKey field
            root.foreignIndex.put(new Key(1001), obj);
        }
        long stop = System.currentTimeMillis();
        time =  stop-start;
        Iterator myIterator = root.strKeyIndex.iterator();
        // iterate through all objects
        int count = 0;
        while (myIterator.hasNext()) 
        {
        	System.out.println("Objects stored: " + count);
        	count++;
        	myIterator.next();
     	}
        Toast.makeText(getApplicationContext(), "Objects stored:" + count, Toast.LENGTH_LONG).show();
        return time;
    }
    
    public ArrayList<MyData> generateFakeData(int anzahl, int byteanzahl)
    {
    	//generiert mit Fakedaten eine Variable von unserem Datentyp
    	ArrayList<MyData> returndata = new ArrayList<MyData>();
    	MyData fakedata;
    	byte blaByte = 1;
    	byte[] dataarray = new byte[byteanzahl];
    	if(byteanzahl > 0)
    	{
    		for(int i = 0; i<byteanzahl; i++)
        	{
        		dataarray[i] = blaByte;
        	}
    	}
    	else
    	{
    		dataarray = null;
    	}
    	for(int i = 0; i < anzahl; i++)
	    {
	    	
	    	fakedata = new MyData();
    		fakedata.intKey = i;
    		fakedata.strKey = "A.B:" + i;
    		fakedata.canID = 100;
    		fakedata.rtr =  blaByte;
    		fakedata.ide = blaByte;
    		fakedata.timestamp=100;
    		fakedata.dlc=byteanzahl;
    		fakedata.data=dataarray;
    		returndata.add(fakedata);
	    }
    	return returndata;
    }
    protected void onStop()
    {
    	if(db.isOpened())
    		db.close();
    	super.onStop();
    }

    protected void onDestroy()
    {
    	if(db.isOpened())
    		db.close();
    	super.onDestroy();
    }
}

package de.gerding.perstperformance;

import org.garret.perst.Persistent;


public class MyData extends Persistent
{
	public int canID;
	public byte rtr; 		
	public byte ide;		
	public int dlc;
	public int timestamp;
	public byte[] data;
	public String strKey;
	public int intKey;
	public String toString() 
	{
		 return intKey + ":" + strKey+ ":" + canID + ":" + Byte.toString(rtr) + ":" + Byte.toString(ide) + ":" + dlc + ":" + timestamp;
	}
}
package de.gerding.perstperformance;

import org.garret.perst.FieldIndex;
import org.garret.perst.Index;
import org.garret.perst.Persistent;
import org.garret.perst.Storage;

public class MyRootClass extends Persistent
{
	// index on MyPersistentClass.intKey
    public FieldIndex<MyData> intKeyIndex;
    // index on MyPersistentClass.strKey
    public FieldIndex<MyData> strKeyIndex;
    // index on MyPersistentClass, which key doesn’t
    // belong to the class
    public Index<MyData> foreignIndex;
    public MyRootClass(Storage db) 
    {
    	 super(db);
    	 intKeyIndex = db.<MyData>createFieldIndex(
        		 MyData.class, // indexed class
        		 "intKey", // name of indexed field
        		 true); // unique index
        strKeyIndex = db.<MyData>createFieldIndex(
       		 MyData.class, // indexed class
       		 "strKey", // name of indexed field
       		 true); // unique index
        foreignIndex = db.<MyData>createIndex(
        		 int.class, // key type
        		true); // unique index)
    }
    // Default constructor is needed for Perst to be able to
    // instantiate instances of loaded objects
    public MyRootClass() {}
}

Share this post


Link to post
Share on other sites

>> I try to save objects of MyData, is my way to do this correct?

I have noticed one wrong or suspicious thing: you are not committing the transaction.

>> Do I have to give own Keys to every object? Is there no way to generate automatic keys?

You do not need to create key object. You can path just integer:

root.foreignIndex.put(i, obj);

Do I have to give own Keys to every object? Is there no way to generate automatic keys?

You have declared index as unique.
Most likely the problem is cause by unique constraint violation: objects with duplicate key values are just not inserted in the index (put method returns false).


Share this post


Link to post
Share on other sites

When I close the app I can reopen and load the data. So I think commit is not important.

I changed the app and know i count the number of objects and then give a new key.

private int countSavedObjects()
    {
    	count = 0;
    	MyData testobj;
        Iterator myIterator = root.intKeyIndex.iterator();
        // iterate through all objects
        while (myIterator.hasNext()) 
        {
        	myIterator.next();
        	count++;
        }
        return count;
    }

Is there a way to get the number of objects stored in the index?

I write my own function, but this is very slow for much objects.

Share this post


Link to post
Share on other sites

Trust us. Commit is important.

>> Is there a way to get the number of objects stored in the index?

Index extends standard Java Collection interface which has size() method. So it is absolutely not needed to iterate through all index members to count number of elements in the index.

Share this post


Link to post
Share on other sites

Trust us. Commit is important.

>> Is there a way to get the number of objects stored in the index?

Index extends standard Java Collection interface which has size() method. So it is absolutely not needed to iterate through all index members to count number of elements in the index.

Thanks for the Information about index.size. it works great. Is it important to commit after ervery insert or is it enough to commit before stop or destroy the app?

Share this post


Link to post
Share on other sites

Another Question about commit.
I will post my complete App Code, there is no commit inside, why I can restart the App and can read the data without making commit?

package de.gerding.perstperformance;

import java.util.ArrayList;
import java.util.Iterator;

import org.garret.perst.Index;
import org.garret.perst.Key;
import org.garret.perst.Persistent;
import org.garret.perst.Storage;
import org.garret.perst.StorageFactory;
import android.app.Activity;
import android.os.Bundle;
import android.view.View;
import android.widget.ArrayAdapter;
import android.widget.EditText;
import android.widget.GridView;
import android.widget.Toast;

public class PerstPerformanceActivity extends Activity
{
	private long time = 0;
	private GridView gridview_times;
	private EditText anzahl_datensaetze;
	private EditText anzahl_datenbytes;
	private ArrayList<String> ergebniszeiten = new ArrayList<String>() ;
	private ArrayAdapter<String> gridviewArrayAdapter;
	private String databaseName = "perstdb";
	final static int pagePoolSize = 2*1024*1024;
	private Storage db;
    private MyRootClass root;
    @Override
    protected void onCreate(Bundle savedInstanceState) 
    {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.perst_performance_activity_layout);
        gridview_times = (GridView) findViewById(R.id.gridView_times);
        anzahl_datensaetze = (EditText) findViewById (R.id.editText_anzahl_datensaetze);
        anzahl_datenbytes = (EditText) findViewById (R.id.editText_anzahl_datenbytes);
        gridviewArrayAdapter = new ArrayAdapter<String>(this, android.R.layout.simple_list_item_1, ergebniszeiten);
 		gridview_times.setAdapter(gridviewArrayAdapter);	
 		ergebniszeiten.add("time in ms");
        ergebniszeiten.add("Datensätze, Datenbytes");
        gridviewArrayAdapter.notifyDataSetChanged(); 
        perstinit();
    }
    public void perstinit()
    {
    	String Filepath = this.getFilesDir().getAbsolutePath();
		// Get instance of Perst storage
        db = StorageFactory.getInstance().createStorage();
        // Open the database with given database name and spacified page pool (database cache) size
        db.open(Filepath+databaseName, pagePoolSize);
        // There is one root object in the database. 
        root = (MyRootClass)db.getRoot();
        if (root == null) 
        {  
        	Toast.makeText(getApplicationContext(), "root Object is null", Toast.LENGTH_LONG).show();
        	// if root object was not specified, then storage is not yet initialized
            // Perform initialization:
            root = new MyRootClass(db);// ... create root object 
            // and register new root object
            db.setRoot(root);
        }     
        Toast.makeText(getApplicationContext(), "Objects stored:" + (root.intKeyIndex.size()), Toast.LENGTH_SHORT).show();
    }
    public void saveFakeData(View v)
    {
    	int anzahl = Integer.valueOf(anzahl_datensaetze.getText().toString());
    	int byteanzahl = Integer.valueOf(anzahl_datenbytes.getText().toString());
    	ArrayList<MyData> fakedata = new ArrayList<MyData>();
    	fakedata= generateFakeData(anzahl, byteanzahl);
    	long minutestart = System.currentTimeMillis();
    	long minutestop = 0;
    	String erg = null ;
    	int docount = 0;
    	do
    	{
    		minutestop = System.currentTimeMillis();
    		erg = perstInsertFakeData(fakedata);
    		ergebniszeiten.add(""+erg);
        	ergebniszeiten.add(""+ anzahl+ ", " + byteanzahl);
        	gridviewArrayAdapter.notifyDataSetChanged();
        	docount++;
    	}while(docount<5);
    	Toast.makeText(getApplicationContext(), "Objects stored:" + (root.intKeyIndex.size()), Toast.LENGTH_SHORT).show();
    }

    private String perstInsertFakeData(ArrayList<MyData> dataarray) 
    {
    	long start = System.currentTimeMillis();
    	long stop = System.currentTimeMillis();
    	int countObjects = 0;
        // Create new object instance
        do
        {
        	MyData obj = dataarray.get(0);
        	obj.intKey = (root.intKeyIndex.size() + 1);
        	obj.strKey = "A.B:" + (root.intKeyIndex.size() + 1);
        	obj.strKey = "TEST";
        	
            // ... and insert it in the corresponding indexes
            root.intKeyIndex.put(obj); // add object to index on intKey field
            //db.commit();
            countObjects++;
            stop = System.currentTimeMillis();
            time =  stop-start;
        }while(time < 1000);
        time =  stop-start;
        String returnstring = "" + countObjects + "||" + time;
        return returnstring;
    }
        
    public ArrayList<MyData> generateFakeData(int anzahl, int byteanzahl)
    {
    	//generiert mit Fakedaten eine Variable von unserem Datentyp, die übergeben Zahl bestimmt die Anzahl der Datenbytes
    	ArrayList<MyData> returndata = new ArrayList<MyData>();
    	MyData fakedata;
    	byte blaByte = 1;
    	byte[] dataarray = new byte[byteanzahl];
    	if(byteanzahl > 0)
    	{
    		for(int i = 0; i<byteanzahl; i++)
        	{
        		dataarray[i] = blaByte;
        	}
    	}
    	else
    	{
    		dataarray = null;
    	}
    	for(int i = 0; i < anzahl; i++)
	    {
	    	
	    	fakedata = new MyData();
    		fakedata.intKey = i;
    		fakedata.strKey = "A.B:" + i;
    		fakedata.canID = 100;
    		fakedata.rtr =  blaByte;
    		fakedata.ide = blaByte;
    		fakedata.timestamp=100;
    		fakedata.dlc=byteanzahl;
    		fakedata.data=dataarray;
    		returndata.add(fakedata);
	    }
    	return returndata;
    }
    
    protected void onStop()
    {
    	if(db.isOpened())
    		db.close();
    	super.onStop();
    }

    protected void onDestroy()
    {
    	if(db.isOpened())
    		db.close();
    	super.onDestroy();
    }
}

package de.gerding.perstperformance;

import org.garret.perst.Persistent;


public class MyData extends Persistent
{
	public int canID;
	public byte rtr; 		
	public byte ide;		
	public int dlc;
	public int timestamp;
	public byte[] data;
	public String strKey;
	public int intKey;
	public String toString() 
	{
		 return intKey + ":" + strKey+ ":" + canID + ":" + Byte.toString(rtr) + ":" + Byte.toString(ide) + ":" + dlc + ":" + timestamp;
	}
}

package de.gerding.perstperformance;

import org.garret.perst.FieldIndex;
import org.garret.perst.Index;
import org.garret.perst.Persistent;
import org.garret.perst.Storage;

public class MyRootClass extends Persistent
{
	// index on MyPersistentClass.intKey
    public FieldIndex<MyData> intKeyIndex;
    // index on MyPersistentClass.strKey
    public FieldIndex<MyData> strKeyIndex;
    // index on MyPersistentClass, which key doesn’t
    // belong to the class
    public Index<MyData> foreignIndex;
    public MyRootClass(Storage db) 
    {
    	 super(db);
    	 intKeyIndex = db.<MyData>createFieldIndex(
        		 MyData.class, // indexed class
        		 "intKey", // name of indexed field
        		 true); // unique index
        strKeyIndex = db.<MyData>createFieldIndex(
       		 MyData.class, // indexed class
       		 "strKey", // name of indexed field
       		 true); // unique index
        foreignIndex = db.<MyData>createIndex(
        		 int.class, // key type
        		true); // unique index)
    }
    // Default constructor is needed for Perst to be able to
    // instantiate instances of loaded objects
    public MyRootClass() {}
}


Share this post


Link to post
Share on other sites

Okay, this explains why the App close ist slowly. Problem is, if I commit after each object, I don't get the performance I need. I need at least 10638 stored objects per second. Thx for your help, I have to discuss this with my team.

Share this post


Link to post
Share on other sites

A transaction is supposed to be "a logical unit of work", in other words, a group of related insert/update/delete operations that you want to succeed or fail all together as an atomic operation.

If you're just bulk loading data, and the definition above doesn't really apply, then you need to try to find the proper balance between transactions that are large enough to sustain your performance objective, but small enough that if the application crashes unexpectedly, or the system shuts down (and Perst doesn't have an opportunity to perform the implicit commit), that you don't lose more data than you can tolerate or recover from in some way.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×