[Pharo-project] [ANN] OpenGL specs parser/FFI methods generator
Hello, i just submitted a code, which could help us greatly in having up-to-date and full OpenGL API coverage in Squeak. This package extracts data directly from spec files, used by OpenGL folks for generating C headers (gl.h / glext.h etc). So it is 100% correct. Parsing these files is much easier than C headers. There's also a lot of additional data , which helps to automatically categorise the methods and generate correct callout code. The parsed data can be later used for generating an FFI/Alien OpenGL callout code with full coverage of all existing extensions. And you don't need to keep this package in image after you done generating the code. Or, it can stay, just make sure that you prune all parsed data by issuing: GLAPIData reset. The sources for parsing is located at: http://www.opengl.org/registry/#specfiles Place all *.spec and *.tm files into a single directory, and then issue: GLAPIData parseAll: (FileDirectory on: 'your/path'). There is an example, how to generate an FFI callout methods. Use: GLAPIData gl generateFFIMethodsInto: (FileStream newFileNamed: 'gl.st') forClass: 'OpenGL' Then, you can file-in the generated code right into your image: (FileStream oldFileNamed: 'gl.st') fileIn The resulting file is about 700kbytes big, having methods like: !OpenGL methodsFor: 'EXT_framebuffer_multisample' stamp: 'Igor.Stasenko 4/14/2010 05:22'! glRenderbufferStorageMultisampleEXT: target with: samples with: internalformat with: width with: height "This method was automatically generated from OpenGL specs" "See http://squeaksource.com/OpenGLSpecs for details" <apicall: void 'glRenderbufferStorageMultisampleEXT' (GLenum GLsizei GLenum GLsizei GLsizei) > ^ self externalCallFailed P.S. I plan to use this data directly by native code, in a way like: glTexCoord2hNV: s with: t <primitive: 'primitiveNativeCall' module: 'NativeBoostPlugin'> ^ self glAPICall: 'glTexCoord2hNV' since i don't need a type data placed in method. I guess, Alien could use similar approach. -- Best regards, Igor Stasenko AKA sig.
On 4/13/2010 8:10 PM, Igor Stasenko wrote:
i just submitted a code, which could help us greatly in having up-to-date and full OpenGL API coverage in Squeak.
This package extracts data directly from spec files, used by OpenGL folks for generating C headers (gl.h / glext.h etc).
Sweet! I had no idea these spec files existed. Do you know if the spec files cover OpenGL ES as well? Also, have you run this against the existing OpenGL code to see if it produces matching output? (more to verify your translation than anything else) I'm definitely interested in looking at this. The original OpenGL calls were generated by code that I've lost in the mean time so it's great to have a way of recreating it from first principles! Cheers, - Andreas
So it is 100% correct. Parsing these files is much easier than C headers. There's also a lot of additional data , which helps to automatically categorise the methods and generate correct callout code.
The parsed data can be later used for generating an FFI/Alien OpenGL callout code with full coverage of all existing extensions. And you don't need to keep this package in image after you done generating the code. Or, it can stay, just make sure that you prune all parsed data by issuing:
GLAPIData reset.
The sources for parsing is located at:
http://www.opengl.org/registry/#specfiles
Place all *.spec and *.tm files into a single directory, and then issue:
GLAPIData parseAll: (FileDirectory on: 'your/path').
There is an example, how to generate an FFI callout methods.
Use:
GLAPIData gl generateFFIMethodsInto: (FileStream newFileNamed: 'gl.st') forClass: 'OpenGL'
Then, you can file-in the generated code right into your image:
(FileStream oldFileNamed: 'gl.st') fileIn
The resulting file is about 700kbytes big, having methods like:
!OpenGL methodsFor: 'EXT_framebuffer_multisample' stamp: 'Igor.Stasenko 4/14/2010 05:22'! glRenderbufferStorageMultisampleEXT: target with: samples with: internalformat with: width with: height "This method was automatically generated from OpenGL specs" "See http://squeaksource.com/OpenGLSpecs for details" <apicall: void 'glRenderbufferStorageMultisampleEXT' (GLenum GLsizei GLenum GLsizei GLsizei)> ^ self externalCallFailed
P.S. I plan to use this data directly by native code, in a way like:
glTexCoord2hNV: s with: t <primitive: 'primitiveNativeCall' module: 'NativeBoostPlugin'> ^ self glAPICall: 'glTexCoord2hNV'
since i don't need a type data placed in method. I guess, Alien could use similar approach.
On 14 April 2010 06:22, Andreas Raab <andreas.raab@gmx.de> wrote:
On 4/13/2010 8:10 PM, Igor Stasenko wrote:
i just submitted a code, which could help us greatly in having up-to-date and full OpenGL API coverage in Squeak.
This package extracts data directly from spec files, used by OpenGL folks for generating C headers (gl.h / glext.h etc).
Sweet! I had no idea these spec files existed. Do you know if the spec files cover OpenGL ES as well? Also, have you run this against the existing OpenGL code to see if it produces matching output? (more to verify your translation than anything else)
I think better would be to check with C header files, like glew (http://sourceforge.net/projects/glew/). The FFI code generator is unfinished (if you look at it, its using an OpenGL-defined standard type set, like GLEnum GLShort etc..) So, there should be 1 more mapping step to turn them into 'long short' etc. Or, if FFI can cope with type aliases, then they could be kept as is. There's also all constant values sitting parsed. But i haven't coded the fileout yet. The repository is open for write globally, so you can just commit your stuff right there. But i'd like to note, that in order to avoid dependencies, it would be better to use separate subpackage for code generation, while in original package there should be just classes for parsing and holding the data.
I'm definitely interested in looking at this. The original OpenGL calls were generated by code that I've lost in the mean time so it's great to have a way of recreating it from first principles!
Yeah. I've been looking for having this a long time :)
Cheers, Â - Andreas
So it is 100% correct. Parsing these files is much easier than C headers. There's also a lot of additional data , which helps to automatically categorise the methods and generate correct callout code.
The parsed data can be later used for generating an FFI/Alien OpenGL callout code with full coverage of all existing extensions. And you don't need to keep this package in image after you done generating the code. Or, it can stay, just make sure that you prune all parsed data by issuing:
GLAPIData reset.
The sources for parsing is located at:
http://www.opengl.org/registry/#specfiles
Place all *.spec and *.tm files into a single directory, and then issue:
 GLAPIData parseAll: (FileDirectory on: 'your/path').
There is an example, how to generate an FFI callout methods.
Use:
 GLAPIData gl generateFFIMethodsInto: (FileStream newFileNamed: 'gl.st') forClass: 'OpenGL'
Then, you can file-in the generated code right into your image:
 (FileStream oldFileNamed: 'gl.st') fileIn
The resulting file is about 700kbytes big, having methods like:
!OpenGL methodsFor: 'EXT_framebuffer_multisample' stamp: 'Igor.Stasenko 4/14/2010 05:22'! glRenderbufferStorageMultisampleEXT: target with: samples with: internalformat with: width with: height     "This method was automatically generated from OpenGL specs"     "See http://squeaksource.com/OpenGLSpecs for details"     <apicall: void 'glRenderbufferStorageMultisampleEXT' (GLenum GLsizei GLenum GLsizei GLsizei)>     ^ self externalCallFailed
P.S. I plan to use this data directly by native code, in a way like:
glTexCoord2hNV: s with: t     <primitive: 'primitiveNativeCall' module: 'NativeBoostPlugin'>     ^ self glAPICall: 'glTexCoord2hNV'
since i don't need a type data placed in method. I guess, Alien could use similar approach.
-- Best regards, Igor Stasenko AKA sig.
On 4/13/2010 8:42 PM, Igor Stasenko wrote:
On 14 April 2010 06:22, Andreas Raab<andreas.raab@gmx.de> wrote:
On 4/13/2010 8:10 PM, Igor Stasenko wrote:
i just submitted a code, which could help us greatly in having up-to-date and full OpenGL API coverage in Squeak.
This package extracts data directly from spec files, used by OpenGL folks for generating C headers (gl.h / glext.h etc).
Sweet! I had no idea these spec files existed. Do you know if the spec files cover OpenGL ES as well? Also, have you run this against the existing OpenGL code to see if it produces matching output? (more to verify your translation than anything else)
I think better would be to check with C header files, like glew (http://sourceforge.net/projects/glew/).
True. Have you tried it?
The FFI code generator is unfinished (if you look at it, its using an OpenGL-defined standard type set, like GLEnum GLShort etc..) So, there should be 1 more mapping step to turn them into 'long short' etc. Or, if FFI can cope with type aliases, then they could be kept as is.
This should be straightforward.
There's also all constant values sitting parsed. But i haven't coded the fileout yet. The repository is open for write globally, so you can just commit your stuff right there.
But i'd like to note, that in order to avoid dependencies, it would be better to use separate subpackage for code generation, while in original package there should be just classes for parsing and holding the data.
Well, I don't really care since this would be (mostly) a one-time job unless someone cares about special API versions ;-) OpenGL should be generated by something like: OpenGLApiGenerator generateFunctions: 1.4 on: OpenGL. which would create the equivalent to the current OpenGL function set. Then we move on to, e.g., OpenGLApiGenerator generateExtFunctions: 'ARB_multisample' on: OpenGL. etc. The default OpenGL would include all versions and extensions but if you'd like to be more specific you can generate custom versions of the API for your use (we'd do this for Teleplace for example since we're trying to stick to 1.4 and certain extensions). Cheers, - Andreas
I'm definitely interested in looking at this. The original OpenGL calls were generated by code that I've lost in the mean time so it's great to have a way of recreating it from first principles!
Yeah. I've been looking for having this a long time :)
Cheers, - Andreas
So it is 100% correct. Parsing these files is much easier than C headers. There's also a lot of additional data , which helps to automatically categorise the methods and generate correct callout code.
The parsed data can be later used for generating an FFI/Alien OpenGL callout code with full coverage of all existing extensions. And you don't need to keep this package in image after you done generating the code. Or, it can stay, just make sure that you prune all parsed data by issuing:
GLAPIData reset.
The sources for parsing is located at:
http://www.opengl.org/registry/#specfiles
Place all *.spec and *.tm files into a single directory, and then issue:
GLAPIData parseAll: (FileDirectory on: 'your/path').
There is an example, how to generate an FFI callout methods.
Use:
GLAPIData gl generateFFIMethodsInto: (FileStream newFileNamed: 'gl.st') forClass: 'OpenGL'
Then, you can file-in the generated code right into your image:
(FileStream oldFileNamed: 'gl.st') fileIn
The resulting file is about 700kbytes big, having methods like:
!OpenGL methodsFor: 'EXT_framebuffer_multisample' stamp: 'Igor.Stasenko 4/14/2010 05:22'! glRenderbufferStorageMultisampleEXT: target with: samples with: internalformat with: width with: height "This method was automatically generated from OpenGL specs" "See http://squeaksource.com/OpenGLSpecs for details" <apicall: void 'glRenderbufferStorageMultisampleEXT' (GLenum GLsizei GLenum GLsizei GLsizei)> ^ self externalCallFailed
P.S. I plan to use this data directly by native code, in a way like:
glTexCoord2hNV: s with: t <primitive: 'primitiveNativeCall' module: 'NativeBoostPlugin'> ^ self glAPICall: 'glTexCoord2hNV'
since i don't need a type data placed in method. I guess, Alien could use similar approach.
On 14 April 2010 06:58, Andreas Raab <andreas.raab@gmx.de> wrote:
On 4/13/2010 8:42 PM, Igor Stasenko wrote:
On 14 April 2010 06:22, Andreas Raab<andreas.raab@gmx.de> Â wrote:
On 4/13/2010 8:10 PM, Igor Stasenko wrote:
i just submitted a code, which could help us greatly in having up-to-date and full OpenGL API coverage in Squeak.
This package extracts data directly from spec files, used by OpenGL folks for generating C headers (gl.h / glext.h etc).
Sweet! I had no idea these spec files existed. Do you know if the spec files cover OpenGL ES as well? Also, have you run this against the existing OpenGL code to see if it produces matching output? (more to verify your translation than anything else)
I think better would be to check with C header files, like glew (http://sourceforge.net/projects/glew/).
True. Have you tried it?
well this might be a bit time consumig: GLAPIData gl functions size 2063 :) so, i think this is going to be a step by step discovery process, for the functions which i will use :)
The FFI code generator is unfinished (if you look at it, its using an OpenGL-defined standard type set, like GLEnum GLShort etc..) So, there should be 1 more mapping step to turn them into 'long short' etc. Or, if FFI can cope with type aliases, then they could be kept as is.
This should be straightforward.
There's also all constant values sitting parsed. But i haven't coded the fileout yet. The repository is open for write globally, so you can just commit your stuff right there.
But i'd like to note, that in order to avoid dependencies, it would be better to use separate subpackage for code generation, while in original package there should be just classes for parsing and holding the data.
Well, I don't really care since this would be (mostly) a one-time job unless someone cares about special API versions ;-)
i don't know, maybe i'm too picky there is FFI and Alien, also i will use it for generating native code callouts, so to not put much mess into classes which holding a data, it would be better to keep code separated, like: FFIOpenGL-Specs AlienOpenGL-Specs NBOpenGL-Specs
OpenGL should be generated by something like:
    OpenGLApiGenerator generateFunctions: 1.4 on: OpenGL.
which would create the equivalent to the current OpenGL function set. Then we move on to, e.g.,
    OpenGLApiGenerator generateExtFunctions: 'ARB_multisample' on: OpenGL.
etc. The default OpenGL would include all versions and extensions but if you'd like to be more specific you can generate custom versions of the API for your use (we'd do this for Teleplace for example since we're trying to stick to 1.4 and certain extensions).
Yeah, i thought about similar approach for filtering a function set by placing them into subclasses like: OpenGL10 -> OpenGL14 -> OpenGL20 ->OpenGL30 etc For instance there is: GetFragDataLocationEXT(program, name) return Int32 param program UInt32 in value param name Char in array [COMPSIZE(name)] category EXT_gpu_shader4 dlflags notlistable version 2.0 extension soft WINSOFT glfflags ignore glxflags ignore alias GetFragDataLocation and GetFragDataLocation(program, name) return Int32 param program UInt32 in value param name Char in array [COMPSIZE(name)] category VERSION_3_0 dlflags notlistable version 3.0 extension glfflags ignore glxflags ignore so, a generator could use a clever approach, like defining a single method #glGetFragDataLocation but depending on driver version, load 'GetFragDataLocationEXT' address for gl 2.0, or 'GetFragDataLocation' for gl 3.0
Cheers, Â - Andreas
-- Best regards, Igor Stasenko AKA sig.
On 14 April 2010 06:22, Andreas Raab <andreas.raab@gmx.de> wrote:
On 4/13/2010 8:10 PM, Igor Stasenko wrote:
i just submitted a code, which could help us greatly in having up-to-date and full OpenGL API coverage in Squeak.
This package extracts data directly from spec files, used by OpenGL folks for generating C headers (gl.h / glext.h etc).
Do you know if the spec files cover OpenGL ES as well?
by looking at gl.spec i see some OES stuff. But its commented out and having different format. So parser don't sees it.
Cheers, Â - Andreas
So it is 100% correct. Parsing these files is much easier than C headers. There's also a lot of additional data , which helps to automatically categorise the methods and generate correct callout code.
The parsed data can be later used for generating an FFI/Alien OpenGL callout code with full coverage of all existing extensions. And you don't need to keep this package in image after you done generating the code. Or, it can stay, just make sure that you prune all parsed data by issuing:
GLAPIData reset.
The sources for parsing is located at:
http://www.opengl.org/registry/#specfiles
Place all *.spec and *.tm files into a single directory, and then issue:
 GLAPIData parseAll: (FileDirectory on: 'your/path').
There is an example, how to generate an FFI callout methods.
Use:
 GLAPIData gl generateFFIMethodsInto: (FileStream newFileNamed: 'gl.st') forClass: 'OpenGL'
Then, you can file-in the generated code right into your image:
 (FileStream oldFileNamed: 'gl.st') fileIn
The resulting file is about 700kbytes big, having methods like:
!OpenGL methodsFor: 'EXT_framebuffer_multisample' stamp: 'Igor.Stasenko 4/14/2010 05:22'! glRenderbufferStorageMultisampleEXT: target with: samples with: internalformat with: width with: height     "This method was automatically generated from OpenGL specs"     "See http://squeaksource.com/OpenGLSpecs for details"     <apicall: void 'glRenderbufferStorageMultisampleEXT' (GLenum GLsizei GLenum GLsizei GLsizei)>     ^ self externalCallFailed
P.S. I plan to use this data directly by native code, in a way like:
glTexCoord2hNV: s with: t     <primitive: 'primitiveNativeCall' module: 'NativeBoostPlugin'>     ^ self glAPICall: 'glTexCoord2hNV'
since i don't need a type data placed in method. I guess, Alien could use similar approach.
-- Best regards, Igor Stasenko AKA sig.
participants (2)
-
Andreas Raab -
Igor Stasenko