Call rpython from outside

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Call rpython from outside

Aleksandr Koshkin
Sup, guys.
I want my rpython function to be invokable from outside world specifically be python. I have wrapped my function with entrypoint_highlevel and it appeared in shared object. So far so good. As a first argument this function takes a pointer to a C struct, and there is a problem. I have precisely recreated this struct in RPython as a lltypes.Struct (not rffi.CStruct) and annotated by this object my entrypoint signature, but it seams that some fields of the passed struct are messed up (shifted basically). Could it be because I used Struct instead of CStruct? I am using CFFI as a binding generator.

--
Kind regards, Aleksandr Koshkin.

_______________________________________________
pypy-dev mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/pypy-dev
Reply | Threaded
Open this post in threaded view
|

Re: Call rpython from outside

Aleksandr Koshkin

2017-07-10 17:03 GMT+03:00 Aleksandr Koshkin <[hidden email]>:
Sup, guys.
I want my rpython function to be invokable from outside world specifically be python. I have wrapped my function with entrypoint_highlevel and it appeared in shared object. So far so good. As a first argument this function takes a pointer to a C struct, and there is a problem. I have precisely recreated this struct in RPython as a lltypes.Struct (not rffi.CStruct) and annotated by this object my entrypoint signature, but it seams that some fields of the passed struct are messed up (shifted basically). Could it be because I used Struct instead of CStruct? I am using CFFI as a binding generator.

--
Kind regards, Aleksandr Koshkin.



--
Kind regards, Aleksandr Koshkin.

_______________________________________________
pypy-dev mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/pypy-dev
Reply | Threaded
Open this post in threaded view
|

Re: Call rpython from outside

Maciej Fijalkowski
lltype.Struct is a GC-managed struct, you don't want to have this as a
part of API (use CStruct)

On Mon, Jul 10, 2017 at 6:15 PM, Aleksandr Koshkin
<[hidden email]> wrote:

> Here is a link to a function that buggs me.
> https://github.com/magniff/rere/blob/master/rere/vm/vm_main.py#L110
> I am using this headers for CFFI:
> https://github.com/magniff/rere/blob/master/rere/build/vm_headers.h
>
> 2017-07-10 17:03 GMT+03:00 Aleksandr Koshkin <[hidden email]>:
>>
>> Sup, guys.
>> I want my rpython function to be invokable from outside world specifically
>> be python. I have wrapped my function with entrypoint_highlevel and it
>> appeared in shared object. So far so good. As a first argument this function
>> takes a pointer to a C struct, and there is a problem. I have precisely
>> recreated this struct in RPython as a lltypes.Struct (not rffi.CStruct) and
>> annotated by this object my entrypoint signature, but it seams that some
>> fields of the passed struct are messed up (shifted basically). Could it be
>> because I used Struct instead of CStruct? I am using CFFI as a binding
>> generator.
>>
>> --
>> Kind regards, Aleksandr Koshkin.
>
>
>
>
> --
> Kind regards, Aleksandr Koshkin.
>
> _______________________________________________
> pypy-dev mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/pypy-dev
>
_______________________________________________
pypy-dev mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/pypy-dev
Reply | Threaded
Open this post in threaded view
|

Re: Call rpython from outside

Maciej Fijalkowski
Sorry wrong, lltype.GcStruct is GC managed, lltype.Struct should work.

However, please use rffi.CStruct (as it's better defined) and
especially rffi.CArray, since lltype.Array contains length field

On Tue, Jul 11, 2017 at 6:49 PM, Maciej Fijalkowski <[hidden email]> wrote:

> lltype.Struct is a GC-managed struct, you don't want to have this as a
> part of API (use CStruct)
>
> On Mon, Jul 10, 2017 at 6:15 PM, Aleksandr Koshkin
> <[hidden email]> wrote:
>> Here is a link to a function that buggs me.
>> https://github.com/magniff/rere/blob/master/rere/vm/vm_main.py#L110
>> I am using this headers for CFFI:
>> https://github.com/magniff/rere/blob/master/rere/build/vm_headers.h
>>
>> 2017-07-10 17:03 GMT+03:00 Aleksandr Koshkin <[hidden email]>:
>>>
>>> Sup, guys.
>>> I want my rpython function to be invokable from outside world specifically
>>> be python. I have wrapped my function with entrypoint_highlevel and it
>>> appeared in shared object. So far so good. As a first argument this function
>>> takes a pointer to a C struct, and there is a problem. I have precisely
>>> recreated this struct in RPython as a lltypes.Struct (not rffi.CStruct) and
>>> annotated by this object my entrypoint signature, but it seams that some
>>> fields of the passed struct are messed up (shifted basically). Could it be
>>> because I used Struct instead of CStruct? I am using CFFI as a binding
>>> generator.
>>>
>>> --
>>> Kind regards, Aleksandr Koshkin.
>>
>>
>>
>>
>> --
>> Kind regards, Aleksandr Koshkin.
>>
>> _______________________________________________
>> pypy-dev mailing list
>> [hidden email]
>> https://mail.python.org/mailman/listinfo/pypy-dev
>>
_______________________________________________
pypy-dev mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/pypy-dev
Reply | Threaded
Open this post in threaded view
|

Re: Call rpython from outside

Aleksandr Koshkin
Thanks for your reply. I have reworked my code a bit - now it uses CStruct instead of Struct.
Now it fails with a rather obscure error https://pastebin.com/MZkni9bU
Anyway, Maciej, see you at PyConRu 17)

2017-07-11 18:20 GMT+03:00 Maciej Fijalkowski <[hidden email]>:
Sorry wrong, lltype.GcStruct is GC managed, lltype.Struct should work.

However, please use rffi.CStruct (as it's better defined) and
especially rffi.CArray, since lltype.Array contains length field

On Tue, Jul 11, 2017 at 6:49 PM, Maciej Fijalkowski <[hidden email]> wrote:
> lltype.Struct is a GC-managed struct, you don't want to have this as a
> part of API (use CStruct)
>
> On Mon, Jul 10, 2017 at 6:15 PM, Aleksandr Koshkin
> <[hidden email]> wrote:
>> Here is a link to a function that buggs me.
>> https://github.com/magniff/rere/blob/master/rere/vm/vm_main.py#L110
>> I am using this headers for CFFI:
>> https://github.com/magniff/rere/blob/master/rere/build/vm_headers.h
>>
>> 2017-07-10 17:03 GMT+03:00 Aleksandr Koshkin <[hidden email]>:
>>>
>>> Sup, guys.
>>> I want my rpython function to be invokable from outside world specifically
>>> be python. I have wrapped my function with entrypoint_highlevel and it
>>> appeared in shared object. So far so good. As a first argument this function
>>> takes a pointer to a C struct, and there is a problem. I have precisely
>>> recreated this struct in RPython as a lltypes.Struct (not rffi.CStruct) and
>>> annotated by this object my entrypoint signature, but it seams that some
>>> fields of the passed struct are messed up (shifted basically). Could it be
>>> because I used Struct instead of CStruct? I am using CFFI as a binding
>>> generator.
>>>
>>> --
>>> Kind regards, Aleksandr Koshkin.
>>
>>
>>
>>
>> --
>> Kind regards, Aleksandr Koshkin.
>>
>> _______________________________________________
>> pypy-dev mailing list
>> [hidden email]
>> https://mail.python.org/mailman/listinfo/pypy-dev
>>



--
Kind regards, Aleksandr Koshkin.

_______________________________________________
pypy-dev mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/pypy-dev
Reply | Threaded
Open this post in threaded view
|

Re: Call rpython from outside

Maciej Fijalkowski
you're missing #includes

On Tue, Jul 11, 2017 at 8:00 PM, Aleksandr Koshkin
<[hidden email]> wrote:

> Thanks for your reply. I have reworked my code a bit - now it uses CStruct
> instead of Struct.
> https://github.com/magniff/rere/blob/master/rere/vm/vm_main.py#L92
> Now it fails with a rather obscure error https://pastebin.com/MZkni9bU
> Anyway, Maciej, see you at PyConRu 17)
>
> 2017-07-11 18:20 GMT+03:00 Maciej Fijalkowski <[hidden email]>:
>>
>> Sorry wrong, lltype.GcStruct is GC managed, lltype.Struct should work.
>>
>> However, please use rffi.CStruct (as it's better defined) and
>> especially rffi.CArray, since lltype.Array contains length field
>>
>> On Tue, Jul 11, 2017 at 6:49 PM, Maciej Fijalkowski <[hidden email]>
>> wrote:
>> > lltype.Struct is a GC-managed struct, you don't want to have this as a
>> > part of API (use CStruct)
>> >
>> > On Mon, Jul 10, 2017 at 6:15 PM, Aleksandr Koshkin
>> > <[hidden email]> wrote:
>> >> Here is a link to a function that buggs me.
>> >> https://github.com/magniff/rere/blob/master/rere/vm/vm_main.py#L110
>> >> I am using this headers for CFFI:
>> >> https://github.com/magniff/rere/blob/master/rere/build/vm_headers.h
>> >>
>> >> 2017-07-10 17:03 GMT+03:00 Aleksandr Koshkin <[hidden email]>:
>> >>>
>> >>> Sup, guys.
>> >>> I want my rpython function to be invokable from outside world
>> >>> specifically
>> >>> be python. I have wrapped my function with entrypoint_highlevel and it
>> >>> appeared in shared object. So far so good. As a first argument this
>> >>> function
>> >>> takes a pointer to a C struct, and there is a problem. I have
>> >>> precisely
>> >>> recreated this struct in RPython as a lltypes.Struct (not
>> >>> rffi.CStruct) and
>> >>> annotated by this object my entrypoint signature, but it seams that
>> >>> some
>> >>> fields of the passed struct are messed up (shifted basically). Could
>> >>> it be
>> >>> because I used Struct instead of CStruct? I am using CFFI as a binding
>> >>> generator.
>> >>>
>> >>> --
>> >>> Kind regards, Aleksandr Koshkin.
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Kind regards, Aleksandr Koshkin.
>> >>
>> >> _______________________________________________
>> >> pypy-dev mailing list
>> >> [hidden email]
>> >> https://mail.python.org/mailman/listinfo/pypy-dev
>> >>
>
>
>
>
> --
> Kind regards, Aleksandr Koshkin.
_______________________________________________
pypy-dev mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/pypy-dev
Reply | Threaded
Open this post in threaded view
|

Re: Call rpython from outside

Aleksandr Koshkin
So ok, I have to specify headers containing my structs and somehow push it to rpython toolchain, if I got you correctly.
0. Why? This structures are already described in the vm file as a bunch of ffi.CStruct objects.
1. If I have to, how would I do that, is there any example of embedding rpython into something? I saw an example "how to embed PyPy" but it looked a bit too PyPy specific.
Thanks

2017-07-11 19:15 GMT+03:00 Maciej Fijalkowski <[hidden email]>:
you're missing #includes

On Tue, Jul 11, 2017 at 8:00 PM, Aleksandr Koshkin
<[hidden email]> wrote:
> Thanks for your reply. I have reworked my code a bit - now it uses CStruct
> instead of Struct.
> https://github.com/magniff/rere/blob/master/rere/vm/vm_main.py#L92
> Now it fails with a rather obscure error https://pastebin.com/MZkni9bU
> Anyway, Maciej, see you at PyConRu 17)
>
> 2017-07-11 18:20 GMT+03:00 Maciej Fijalkowski <[hidden email]>:
>>
>> Sorry wrong, lltype.GcStruct is GC managed, lltype.Struct should work.
>>
>> However, please use rffi.CStruct (as it's better defined) and
>> especially rffi.CArray, since lltype.Array contains length field
>>
>> On Tue, Jul 11, 2017 at 6:49 PM, Maciej Fijalkowski <[hidden email]>
>> wrote:
>> > lltype.Struct is a GC-managed struct, you don't want to have this as a
>> > part of API (use CStruct)
>> >
>> > On Mon, Jul 10, 2017 at 6:15 PM, Aleksandr Koshkin
>> > <[hidden email]> wrote:
>> >> Here is a link to a function that buggs me.
>> >> https://github.com/magniff/rere/blob/master/rere/vm/vm_main.py#L110
>> >> I am using this headers for CFFI:
>> >> https://github.com/magniff/rere/blob/master/rere/build/vm_headers.h
>> >>
>> >> 2017-07-10 17:03 GMT+03:00 Aleksandr Koshkin <[hidden email]>:
>> >>>
>> >>> Sup, guys.
>> >>> I want my rpython function to be invokable from outside world
>> >>> specifically
>> >>> be python. I have wrapped my function with entrypoint_highlevel and it
>> >>> appeared in shared object. So far so good. As a first argument this
>> >>> function
>> >>> takes a pointer to a C struct, and there is a problem. I have
>> >>> precisely
>> >>> recreated this struct in RPython as a lltypes.Struct (not
>> >>> rffi.CStruct) and
>> >>> annotated by this object my entrypoint signature, but it seams that
>> >>> some
>> >>> fields of the passed struct are messed up (shifted basically). Could
>> >>> it be
>> >>> because I used Struct instead of CStruct? I am using CFFI as a binding
>> >>> generator.
>> >>>
>> >>> --
>> >>> Kind regards, Aleksandr Koshkin.
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Kind regards, Aleksandr Koshkin.
>> >>
>> >> _______________________________________________
>> >> pypy-dev mailing list
>> >> [hidden email]
>> >> https://mail.python.org/mailman/listinfo/pypy-dev
>> >>
>
>
>
>
> --
> Kind regards, Aleksandr Koshkin.



--
Kind regards, Aleksandr Koshkin.

_______________________________________________
pypy-dev mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/pypy-dev
Reply | Threaded
Open this post in threaded view
|

Re: Call rpython from outside

Armin Rigo-2
Hi Aleksandr,

On 11 July 2017 at 18:33, Aleksandr Koshkin <[hidden email]> wrote:
> So ok, I have to specify headers containing my structs and somehow push it
> to rpython toolchain, if I got you correctly.
> 0. Why? This structures are already described in the vm file as a bunch of
> ffi.CStruct objects.

rffi.CStruct() is used to declare the RPython interface for structs
that are originally defined in C.

You can use lltype.Struct(), but it's not recommended in your case
because lltype.Struct() is meant to define structs in RPython where
you *don't* need a precise C-level struct; for example,
lltype.Struct() could rename and reorder the fields in C if it is more
efficient.

We don't have a direct way to declare the struct in RPython but also
force it to generate exactly the C struct declaration you want,
because we never needed it.  You need to use rffi.CStruct() and write
the struct in the .h file manually too.

> 1. If I have to, how would I do that, is there any example of embedding
> rpython into something?

Not really.  Look maybe at tests using rpython.rlib.entrypoint, like
rpython/translator/c/test/test_genc:test_entrypoints.


A bientôt,

Armin.
_______________________________________________
pypy-dev mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/pypy-dev
Reply | Threaded
Open this post in threaded view
|

Re: Call rpython from outside

Maciej Fijalkowski
Hi Armin

We ended up (Aleksandr is here at pycon russia) using rffi_platform to
get the exact shape of the structure from the header file. There were
a few bugs how exactly this got mapped, so it ended up being a good
way to do it.

On Sat, Jul 15, 2017 at 8:02 AM, Armin Rigo <[hidden email]> wrote:

> Hi Aleksandr,
>
> On 11 July 2017 at 18:33, Aleksandr Koshkin <[hidden email]> wrote:
>> So ok, I have to specify headers containing my structs and somehow push it
>> to rpython toolchain, if I got you correctly.
>> 0. Why? This structures are already described in the vm file as a bunch of
>> ffi.CStruct objects.
>
> rffi.CStruct() is used to declare the RPython interface for structs
> that are originally defined in C.
>
> You can use lltype.Struct(), but it's not recommended in your case
> because lltype.Struct() is meant to define structs in RPython where
> you *don't* need a precise C-level struct; for example,
> lltype.Struct() could rename and reorder the fields in C if it is more
> efficient.
>
> We don't have a direct way to declare the struct in RPython but also
> force it to generate exactly the C struct declaration you want,
> because we never needed it.  You need to use rffi.CStruct() and write
> the struct in the .h file manually too.
>
>> 1. If I have to, how would I do that, is there any example of embedding
>> rpython into something?
>
> Not really.  Look maybe at tests using rpython.rlib.entrypoint, like
> rpython/translator/c/test/test_genc:test_entrypoints.
>
>
> A bientôt,
>
> Armin.
_______________________________________________
pypy-dev mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/pypy-dev
Reply | Threaded
Open this post in threaded view
|

Re: Call rpython from outside

Aleksandr Koshkin
Great! 

16 июл. 2017 г. 2:32 ПП пользователь "Maciej Fijalkowski" <[hidden email]> написал:
Hi Armin

We ended up (Aleksandr is here at pycon russia) using rffi_platform to
get the exact shape of the structure from the header file. There were
a few bugs how exactly this got mapped, so it ended up being a good
way to do it.

On Sat, Jul 15, 2017 at 8:02 AM, Armin Rigo <[hidden email]> wrote:
> Hi Aleksandr,
>
> On 11 July 2017 at 18:33, Aleksandr Koshkin <[hidden email]> wrote:
>> So ok, I have to specify headers containing my structs and somehow push it
>> to rpython toolchain, if I got you correctly.
>> 0. Why? This structures are already described in the vm file as a bunch of
>> ffi.CStruct objects.
>
> rffi.CStruct() is used to declare the RPython interface for structs
> that are originally defined in C.
>
> You can use lltype.Struct(), but it's not recommended in your case
> because lltype.Struct() is meant to define structs in RPython where
> you *don't* need a precise C-level struct; for example,
> lltype.Struct() could rename and reorder the fields in C if it is more
> efficient.
>
> We don't have a direct way to declare the struct in RPython but also
> force it to generate exactly the C struct declaration you want,
> because we never needed it.  You need to use rffi.CStruct() and write
> the struct in the .h file manually too.
>
>> 1. If I have to, how would I do that, is there any example of embedding
>> rpython into something?
>
> Not really.  Look maybe at tests using rpython.rlib.entrypoint, like
> rpython/translator/c/test/test_genc:test_entrypoints.
>
>
> A bientôt,
>
> Armin.

_______________________________________________
pypy-dev mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/pypy-dev