freelanceprogrammers.org Forum Index » Delphi

View previous topic :: View next topic

which type of var-interface to use ?

aap_beertje Posted: Sat Dec 30, 2006 11:34 pm

Joined: 30 Dec 2006

Posts: 15

which type of var-interface to use ?

hi,

I`ve just made my first Python/SciPy program,
and my first impressions are very good!

I`ve a real-time application, currently written in Delphi,
with embedded MatLab for the number crunching.
Now I want to replace the MatLab OLE component,
with an interface to Python.
I think that should be possible,
the number crunching I need is much faster in Python than in MatLab,
so that solves my first doubt.

Now the embedding Python part,
I read several documents,
but can not get a clear picture of what kind of interface to use,
so maybe someone can give som tips,
instead of trying to test all the 30 examples.

My application needs the following:
1. some semi-static data settings, about 200 doubles, which could be
modified by both the Delphi part and the Python part.
2. the real time streaming part, where streaming is done in chunks of
data, each chunk
- Delphi to Python: array [0..7, 0..999] of double
- run some preloaded (together with the settings) Python script
- Python to Delphi return: array [0..15, 0..999] of double

What var-type should I use ?
Are there any problems to be expected with this setup in Python ?

thanks,
Stef Mientki

aap_beertje Posted: Sun Dec 31, 2006 12:46 am

Joined: 30 Dec 2006

Posts: 15

which type of var-interface to use ?

oliverbock Posted: Sun Dec 31, 2006 7:10 am

Joined: 15 Sep 2005

Posts: 12

which type of var-interface to use ?

Hi Stef,

aap_beertje wrote:
> My application needs the following:
> 1. some semi-static data settings, about 200 doubles, which could be
> modified by both the Delphi part and the Python part.
>

I would use PyEngine to construct a Python dict (PyDict_New) stored as a
PPyObject (not a variant) and put your settings in there (PyDict_SetXxx,
PyDict_GetXxx). Of course a Python dict will be easy to access/modify
from within Python, although you must find some way to pass it in (as a
function parameter, or in the global or local variables dictionary).
From Delphi you can build provide get/set code using setting names. If
performance from Delphi is important then you can speed up lookup by
pre-building Python strings for each setting name. Don`t forget to call
Py_DECREF on your dictionary when you`re finished with it.

> 2. the real time streaming part, where streaming is done in chunks of
> data, each chunk
> - Delphi to Python: array [0..7, 0..999] of double
> - run some preloaded (together with the settings) Python script
> - Python to Delphi return: array [0..15, 0..999] of double
>
> What var-type should I use ?
> Are there any problems to be expected with this setup in Python ?
>
Presumably performance is important here, so of course you should be
using raw PPyObjects again. You can find all the function you need by
looking through TPythonEngine. You need to get reference counts right
so take a look at http://docs.python.org/ext/refcounts.html

However you still wouldn`t be wasting your time to go through the
samples. I would have saved myself some time if I`d bothered doing that
first!

Oliver

Oliver

aap_beertje Posted: Sun Dec 31, 2006 4:59 pm

Joined: 30 Dec 2006

Posts: 15

which type of var-interface to use ?

hi Oliver,

> > 2. the real time streaming part, where streaming is done in chunks of
> > data, each chunk
> > - Delphi to Python: array [0..7, 0..999] of double
> > - run some preloaded (together with the settings) Python script
> > - Python to Delphi return: array [0..15, 0..999] of double
> >
> > What var-type should I use ?
> > Are there any problems to be expected with this setup in Python ?
> >
> Presumably performance is important here, so of course you should be
> using raw PPyObjects again. You can find all the function you need by
> looking through TPythonEngine. You need to get reference counts right
> so take a look at http://docs.python.org/ext/refcounts.html
>
Thanks very much for the pointers, I`ll start studying TPythonEngine
and refcounts.

> However you still wouldn`t be wasting your time to go through the
> samples. I would have saved myself some time if I`d bothered doing
that
> first!
I guess that depends on what your goal is,
I just need about 10 .. 20 lines of code ONCE,
to get Python embedded in almost the SAME way as MatLab embeddeding is
done, so I can convince my collegues that Python is very good
alternative for MatLab. (MatLab has just one OLE interface it can only
exchange 2-dimensional arrays of doubles)
So studying (not just looking at them) all the beautiful concepts in
the examples (without any knowledge of VarPyth, PythonDelphiVar,
Variants, OOPs, bindings etc), would be a waste of time, the time I
prefer to investigate in Python ;-). Btw. I was really really
impressed by example 31, (the creation and use of Delphi components
from within Python), never thought such a construct would be possible,
very good, but I don`t have an application for it now :-(

thanks again,
cheers,
Stef Mientki

morgan_martinet Posted: Sun Dec 31, 2006 9:09 pm

Joined: 29 Apr 2005

Posts: 103

which type of var-interface to use ?

Hi Stef,

You can also investigate the Python module NumPy
(http://numpy.scipy.org/) that is dedicated to handling large arrays
and matrixes...

Another way, if you want the best performance, would be to extend
Python with P4D and provide your own new type that would contain your
array [0..7, 0..999] of double. That way you would have a direct
access to the array from Delphi and you could expose getter/setter for
Python to access items. Of course this would require more
work/knowledge than a 30 lines of code.

Also, if you don`t want to spend too much time with reference counting
then you should stick with VarPyth.pas which abstracts this. The
drawback is that it relies on custom variants and thus adds some
overhead compared to using native Python APIs directly.
Lood at Demo25 to understand all you can do with VarPyth. Note that
once you understand Python a little more, it`s fairly easy to convert
a Python script directly into a VarPyth dialect which would lets you
handle Python object directly from Delphi, as you would do it from
Python itself.

Bye,

Morgan

--- In pythonfordelphi@yahoogroups.com, "aap_beertje" <s.mientki@...>
wrote:
>
> hi Oliver,
>
> > > 2. the real time streaming part, where streaming is done in
chunks of
> > > data, each chunk
> > > - Delphi to Python: array [0..7, 0..999] of double
> > > - run some preloaded (together with the settings) Python script
> > > - Python to Delphi return: array [0..15, 0..999] of double
> > >
> > > What var-type should I use ?
> > > Are there any problems to be expected with this setup in Python ?
> > >
> > Presumably performance is important here, so of course you should be
> > using raw PPyObjects again. You can find all the function you
need by
> > looking through TPythonEngine. You need to get reference counts
right
> > so take a look at http://docs.python.org/ext/refcounts.html
> >
> Thanks very much for the pointers, I`ll start studying TPythonEngine
> and refcounts.
>
> > However you still wouldn`t be wasting your time to go through the
> > samples. I would have saved myself some time if I`d bothered doing
> that
> > first!
> I guess that depends on what your goal is,
> I just need about 10 .. 20 lines of code ONCE,
> to get Python embedded in almost the SAME way as MatLab embeddeding is
> done, so I can convince my collegues that Python is very good
> alternative for MatLab. (MatLab has just one OLE interface it can only
> exchange 2-dimensional arrays of doubles)
> So studying (not just looking at them) all the beautiful concepts in
> the examples (without any knowledge of VarPyth, PythonDelphiVar,
> Variants, OOPs, bindings etc), would be a waste of time, the time I
> prefer to investigate in Python ;-). Btw. I was really really
> impressed by example 31, (the creation and use of Delphi components
> from within Python), never thought such a construct would be possible,
> very good, but I don`t have an application for it now :-(
>
> thanks again,
> cheers,
> Stef Mientki
>

kxroberto Posted: Mon Jan 01, 2007 12:58 am

Joined: 21 Dec 2006

Posts: 8

which type of var-interface to use ?

--- In pythonfordelphi@yahoogroups.com, "Morgan Martinet" <yahoo@...>
wrote:
>
> Hi Stef,
>
> You can also investigate the Python module NumPy
> (http://numpy.scipy.org/) that is dedicated to handling large arrays
> and matrixes...
>
> Another way, if you want the best performance, would be to extend
> Python with P4D and provide your own new type that would contain your
> array [0..7, 0..999] of double. That way you would have a direct
> access to the array from Delphi and you could expose getter/setter for
> Python to access items. Of course this would require more
> work/knowledge than a 30 lines of code.

Or somewhat dirty, but very fast and effective:
Do around with numpy/scipy. Make numpy arrays contigious by
".flatten()" etc. or even ".tostring()" and simply cast the array
memory start address directly to a Pascal Array pointer - while not
loosing a last holding ref.
Do simple/non-time-critical management things - usually >95% of the
effort - with VarPyth and PythonCallback.

And don`t forget: Use profiler evidence over mere tech imagination in
order to see how rare the use cases are for optimization on todays
computers! You`d usually just pump some compressed (graphical?)
output data back to Delphi etc.. This is not much. For many use cases
7000 items are near to nothing and just non-optimized dummy iteration
does it already...

Robert

aap_beertje Posted: Mon Jan 01, 2007 4:42 pm

Joined: 30 Dec 2006

Posts: 15

which type of var-interface to use ?

hi Morgan,

> Also, if you don`t want to spend too much time with reference counting
> then you should stick with VarPyth.pas which abstracts this. The
> drawback is that it relies on custom variants and thus adds some
> overhead compared to using native Python APIs directly.
> Lood at Demo25 to understand all you can do with VarPyth. Note that
> once you understand Python a little more, it`s fairly easy to convert
> a Python script directly into a VarPyth dialect which would lets you
> handle Python object directly from Delphi, as you would do it from
> Python itself.

I think, that will be the first way to go, keep it standard en simple,
worry about the speed later.
thanks for the hint,
Stef Mientki

> > Stef Mientki
> >
>

aap_beertje Posted: Mon Jan 01, 2007 4:50 pm

Joined: 30 Dec 2006

Posts: 15

which type of var-interface to use ?

hi Robert,

> > Another way, if you want the best performance, would be to extend
> > Python with P4D and provide your own new type that would contain your
> > array [0..7, 0..999] of double. That way you would have a direct
> > access to the array from Delphi and you could expose getter/setter for
> > Python to access items. Of course this would require more
> > work/knowledge than a 30 lines of code.
>
> Or somewhat dirty, but very fast and effective:
> Do around with numpy/scipy. Make numpy arrays contigious by
> ".flatten()" etc. or even ".tostring()" and simply cast the array
> memory start address directly to a Pascal Array pointer - while not
> loosing a last holding ref.
That might be a good idea (at least for the future), I saw the picture
rotating demo, where the picture information is passed as a string.

> Do simple/non-time-critical management things - usually >95% of the
> effort - with VarPyth and PythonCallback.
>
> And don`t forget: Use profiler evidence over mere tech imagination in
> order to see how rare the use cases are for optimization on todays
> computers! You`d usually just pump some compressed (graphical?)
> output data back to Delphi etc.. This is not much. For many use cases
> 7000 items are near to nothing and just non-optimized dummy iteration
> does it already...
Yes you`re right, first worry about the principle, and then, only if
absoluteley needed, improve the speed.
BTW, it`s not a graphical job, the idea is to sample 8 analog channels
in 20 bit, at 1000 Hz, and process the data in real time, not only
compressing, but extracting more features from it and therefor it will
expand in 16 channels (20 bit / 1000Hz) of calculated signals. And the
target is medical research in an academical hospital.

thanks,
Stef Mientki

freelanceprogrammers.org Forum Index » Delphi

All times are GMT
Page 1 of 1

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

Freelace Website Designer - Customer web design and software building.
English Courses in England - reservation calendar script
Land Surveying -Stonex instruments and equipments
China Wholesale - Electronics Products
Character Studio - Tutorials and Help