Difference between revisions of "Silme:Tutorial:IOClient"

From Braniecki's Wiki
Jump to navigation Jump to search
 
(2 intermediate revisions by the same user not shown)
Line 1: Line 1:
As you probably noticed, until now we did not get into any way to actually operate on the l10n objects outside of memory. By this moment you should know how to create an Entity, Comment, how to create an EntityList, L10nObject and L10nPackage and bundle it all together.
+
As you probably noticed, until now we did not get into any way to actually operate on the l10n objects outside of memory. At this point you should know how to create an Entity, Comment, how to create an EntityList, L10nObject and L10nPackage, traverse, join and search and bundle it all together.
  
 
Now we'll focus on the ways you can use to load/save those objects to actually be able to use the system in the real life.
 
Now we'll focus on the ways you can use to load/save those objects to actually be able to use the system in the real life.
  
There are two levels of this operations. First one is an actual loading entities from somewhere to internal structure like EntityList, L10nObject or L10nPackage and saving it to some place. This is covered by IOClient family.
+
There are two levels of this operations. First one is an actual loading entities from somewhere to internal structure like EntityList, L10nObject or L10nPackage and saving it to some place. This is covered by '''silme.io''' family.
  
== IOManager ==
+
== silme.io ==
  
IOManager class is a command center for all kinds of IOClients classes. It means that you should use it to load a proper I/O class. Currently, the project supports several I/O's like '''file''', '''svn''', '''sqlite''', '''jar'''.
+
silme.io is a package that contains everything needed to read and write from different sources including - file system, databases and revision control systems.
  
 +
Each separate class that manages reading and writing is called a Client. There are several abstract clients bundled in silme.io.clients that are bases for all other clients. For example there is IOClient, DBCLient and RCSClient which are bases for FileClient, MysqlClient, SVNClient respectively.
 +
 +
The clients are managed by silme.io.Manager which can register and select best client for us.
  
 
== IOClient ==
 
== IOClient ==
  
 
IOClient is an abstract class that is a base for every I/O client that can read and write objects. For example a FileClient can read a file into memory and write it back to the hard drive. Or it can load directory as L10nPackage. SvnClient can load L10nPackage from the directory in SVN repository, JarClient can load a '''*.jar''' file as L10nPackage and SQLiteClient can read tables as L10nObjects.
 
IOClient is an abstract class that is a base for every I/O client that can read and write objects. For example a FileClient can read a file into memory and write it back to the hard drive. Or it can load directory as L10nPackage. SvnClient can load L10nPackage from the directory in SVN repository, JarClient can load a '''*.jar''' file as L10nPackage and SQLiteClient can read tables as L10nObjects.
 +
 +
== silme.io.Manager ==
 +
 +
silme.io.Manager has 3 important methods:
 +
* register() - allows you to register clients in a form: silme.io.Manager.register('file', 'jar', 'mysql')
 +
* register_all() - registers all clients it will find in its path
 +
* get(name=) - allows you to get client (if it's not loaded and Manager will find it, it'll load it automatically)
 +
* get(path=) - will test each registered client to check if it supports this kind of path and return first that does
 +
  
 
=== Reading ===
 
=== Reading ===
Line 19: Line 31:
  
 
<code lang="python">
 
<code lang="python">
from mozilla.l10n.object import L10nObject, L10nPackage
+
import silme.core
from mozilla.l10n.io.IOManager import IOManager
+
import silme.io
from mozilla.l10n.io.file import FileClient
+
 
from mozilla.l10n.io.jar import JarClient
+
silme.io.Manager.register('file', 'jar')
 +
 
 +
ioclient = silme.io.Manager.get('file') # you can you other types here like: jar, sqlite, mysql, svn
 +
l10npackage = ioclient.get_l10npackage(path='./test/')
  
ioClient = IOManager.get('file') # you can you other types here like: jar, sqlite, mysql, svn
 
l10nPackage = ioClient.getL10nPackage(path='./test/')
 
  
 +
l10nobject = ioclient.get_l10nobject(path='./test/example.dtd')
  
l10nObject = ioClient.getL10nObject(path='./test/example.dtd')
+
# Manager fires client.matches_path(path) on each client and returns the one which will return True first
 +
ioclient2 = silme.io.Manager.get(path='./data/test.jar')
  
ioClient2 = IOManager.get('jar')
+
l10npackage2 = ioclient2.get_l10npackage(path='./test.jar')
l10nPackage2 = ioClient.getL10nPackage(path='./test.jar')
 
 
</code>
 
</code>
  
In result we have two l10nPackages, one from the directory on the hard drive, and another extracted from the .jar file.
+
In result we have two L10nPackages, one from the directory on the hard drive, and another extracted from the .jar file.
Also, we have an l10nObject from the hard drive.
+
Also, we have an L10nObject from the hard drive.
  
  
Line 43: Line 57:
  
 
<code lang="python">
 
<code lang="python">
from mozilla.l10n.object import L10nObject, L10nPackage
+
import silme.core
from mozilla.l10n.io.IOManager import IOManager
+
import silme.io
from mozilla.l10n.io.file import FileClient
 
  
ioClient = IOManager.get('file')
+
silme.io.Manager.register('file')
ioClient.writeL10nPackage(l10nPackage, path='../newpath')
 
  
ioClient.writeObject(l10nObject, path='../../example.dtd')
+
ioclient = IOManager.get('file')
 +
ioclient.write_l10npackage(l10npack, path='../newpath')
 +
 
 +
ioclient.write_object(l10nobject, path='../../example.dtd')
  
 
</code>
 
</code>
Line 56: Line 71:
 
IOClient lets you read and write between the objects in the memory of the program and an actual storage of the data. It may be a database, hard drive, zip package or an RCS.
 
IOClient lets you read and write between the objects in the memory of the program and an actual storage of the data. It may be a database, hard drive, zip package or an RCS.
  
== modifying l10nObject ==
+
== It's getting exciting! ==
  
Thanks to what we already know we can now load an L10nObject, modify it an save back to the file:
+
Thanks to what we already know we can now (almost) load an L10nObject, modify it an save back to the file:
  
 
<code lang="python">
 
<code lang="python">
from mozilla.l10n.object import L10nObject
+
import silme.core
from mozilla.l10n.io.IOManager import IOManager
+
import silme.io
from mozilla.l10n.io.file import FileClient
 
  
 
file = './test/example.dtd'
 
file = './test/example.dtd'
  
ioClient = IOManager.get('file')
+
ioclient = silme.io.Manager.get('file')
  
l10nObject = ioClient.getL10nObject(path=file)
+
l10nobject = ioclient.get_l10nobject(path=file)
l10nObject.modifyEntity('entity.id', u'new value')
+
 
ioClient.writeL10nObject(l10nObject, path=file)
+
l10nobject.modify_entity('entity.id', u'new value')
 +
 
 +
ioclient.write_l10nobject(l10nobject, path=file)
 
</code>
 
</code>
  
This is a very convenient way to work with L10nObjects. You can of course easily operate on all elements of nested L10nPackage structure, modify entities inside and then save the whole package somewhere.
+
This is a very convenient way to work with L10nObjects. You can of course easily operate on all elements of nested L10nPackage structure, modify entities inside, move them around and then save the whole package somewhere.
  
 
== Extended ==
 
== Extended ==
 
=== operating on entities without loading file to memory ===
 
 
In other cases, you may decide that it's not worth loading whole L10nObject to memory just to change/add/remove entity.
 
 
For such cases the library offers ability to perform three simple operations on the object in path without loading it to the memory:
 
 
<code lang="python">
 
from mozilla.l10n.object import L10nObject
 
from mozilla.l10n.io.IOManager import IOManager
 
from mozilla.l10n.io.file import FileClient
 
 
file = './test/example.dtd'
 
 
ioClient = IOManager.get('file')
 
 
(...)
 
</code>
 
  
 
=== SQLClient specifics ===
 
=== SQLClient specifics ===
Line 111: Line 109:
  
 
Second, when you save the data, you have to decide how you want the changes to be saves. Should there be a commit on every single change or would you like to offer a user a dynamic sandbox with his changes that are commited once in a while.
 
Second, when you save the data, you have to decide how you want the changes to be saves. Should there be a commit on every single change or would you like to offer a user a dynamic sandbox with his changes that are commited once in a while.
 +
 +
In the [[Silme:Tutorial:FormatParser|next chapter]] we will get a better understanding on what actually happens during read and write of the object and how silme knows which format to use.

Latest revision as of 14:13, 16 February 2009

As you probably noticed, until now we did not get into any way to actually operate on the l10n objects outside of memory. At this point you should know how to create an Entity, Comment, how to create an EntityList, L10nObject and L10nPackage, traverse, join and search and bundle it all together.

Now we'll focus on the ways you can use to load/save those objects to actually be able to use the system in the real life.

There are two levels of this operations. First one is an actual loading entities from somewhere to internal structure like EntityList, L10nObject or L10nPackage and saving it to some place. This is covered by silme.io family.

silme.io

silme.io is a package that contains everything needed to read and write from different sources including - file system, databases and revision control systems.

Each separate class that manages reading and writing is called a Client. There are several abstract clients bundled in silme.io.clients that are bases for all other clients. For example there is IOClient, DBCLient and RCSClient which are bases for FileClient, MysqlClient, SVNClient respectively.

The clients are managed by silme.io.Manager which can register and select best client for us.

IOClient

IOClient is an abstract class that is a base for every I/O client that can read and write objects. For example a FileClient can read a file into memory and write it back to the hard drive. Or it can load directory as L10nPackage. SvnClient can load L10nPackage from the directory in SVN repository, JarClient can load a *.jar file as L10nPackage and SQLiteClient can read tables as L10nObjects.

silme.io.Manager

silme.io.Manager has 3 important methods:

  • register() - allows you to register clients in a form: silme.io.Manager.register('file', 'jar', 'mysql')
  • register_all() - registers all clients it will find in its path
  • get(name=) - allows you to get client (if it's not loaded and Manager will find it, it'll load it automatically)
  • get(path=) - will test each registered client to check if it supports this kind of path and return first that does


Reading

In the simple case you just use IOManager to load a proper IOClient and then interact with selected type of IO. For example:

import silme.core import silme.io

silme.io.Manager.register('file', 'jar')

ioclient = silme.io.Manager.get('file') # you can you other types here like: jar, sqlite, mysql, svn l10npackage = ioclient.get_l10npackage(path='./test/')


l10nobject = ioclient.get_l10nobject(path='./test/example.dtd')

  1. Manager fires client.matches_path(path) on each client and returns the one which will return True first

ioclient2 = silme.io.Manager.get(path='./data/test.jar')

l10npackage2 = ioclient2.get_l10npackage(path='./test.jar')

In result we have two L10nPackages, one from the directory on the hard drive, and another extracted from the .jar file. Also, we have an L10nObject from the hard drive.


Writing

To write down an L10nObject or L10nPackage you simply use the same IOClient:

import silme.core import silme.io

silme.io.Manager.register('file')

ioclient = IOManager.get('file') ioclient.write_l10npackage(l10npack, path='../newpath')

ioclient.write_object(l10nobject, path='../../example.dtd')

IOClient lets you read and write between the objects in the memory of the program and an actual storage of the data. It may be a database, hard drive, zip package or an RCS.

It's getting exciting!

Thanks to what we already know we can now (almost) load an L10nObject, modify it an save back to the file:

import silme.core import silme.io

file = './test/example.dtd'

ioclient = silme.io.Manager.get('file')

l10nobject = ioclient.get_l10nobject(path=file)

l10nobject.modify_entity('entity.id', u'new value')

ioclient.write_l10nobject(l10nobject, path=file)

This is a very convenient way to work with L10nObjects. You can of course easily operate on all elements of nested L10nPackage structure, modify entities inside, move them around and then save the whole package somewhere.

Extended

SQLClient specifics

It's important to remember that different kinds of IOClient will give different results. For example SQLClients will not be able to produce full L10nObject and they will serve EntityLists instead.

The reason is that L10nObject is in fact EntityList with a textual context, while SQL does not have any context beside of table rows that represent data.

In most situations you will want to extend the generic SQLClient to suite it to your database structure. You may want to store each locale as a separate column in one table or as a separate table.

RCSClient specifics

In case of RCSCLients their nature requires a user to slightly change the way he operates with the data.

First, while loading L10nObject or L10nPackage, you can select revision from which you want the data to be.

Second, when you save the data, you have to decide how you want the changes to be saves. Should there be a commit on every single change or would you like to offer a user a dynamic sandbox with his changes that are commited once in a while.

In the next chapter we will get a better understanding on what actually happens during read and write of the object and how silme knows which format to use.