diff options
Diffstat (limited to 'emulator/opengl/DESIGN')
| -rw-r--r-- | emulator/opengl/DESIGN | 614 |
1 files changed, 0 insertions, 614 deletions
diff --git a/emulator/opengl/DESIGN b/emulator/opengl/DESIGN deleted file mode 100644 index 70a5d2733..000000000 --- a/emulator/opengl/DESIGN +++ /dev/null @@ -1,614 +0,0 @@ -Android Hardware OpenGLES emulation design overview -=================================================== - -Introduction: -------------- - -Hardware GLES emulation in the Android platform is implemented with a mix -of components, which are: - - - Several host "translator" libraries. They implement the EGL, GLES 1.1 and - GLES 2.0 ABIs defined by Khronos, and translate the corresponding function - calls into calls to the appropriate desktop APIs, i.e.: - - - Xgl (Linux), AGL (OS X) or WGL (Windows) for EGL - - desktop GL 2.0 for GLES 1.1 and GLES 2.0 - - _________ __________ __________ - | | | | | | HOST - |TRANSLATOR |TRANSLATOR| |TRANSLATOR| HOST - | EGL | | GLES 1.1 | | GLES 2.0 | TRANSLATOR - |_________| |__________| |__________| LIBRARIES - | | | - - - - | - - - - - - - - - | - - - - - - - - - | - - - - - - | | | - ____v____ ____v_____ _____v____ HOST - | | | | | | SYSTEM - | Xgl | | GL 2.0 | | GL 2.0 | LIBRARIES - |_________| |__________| |__________| - - - - - Several system libraries inside the emulated guest system that implement - the same EGL / GLES 1.1 and GLES 2.0 ABIs. - - They collect the sequence of EGL/GLES function calls and translate then - into a custom wire protocol stream that is sent to the emulator program - through a high-speed communication channel called a "QEMU Pipe". - - For now, all you need to know is that the pipe is implemented with a - custom kernel driver, and provides for _very_ fast bandwidth. All read() - and writes() from/to the pipes are essentially instantaneous from the - guest's point of view. - - - _________ __________ __________ - | | | | | | - |EMULATION| |EMULATION | |EMULATION | GUEST - | EGL | | GLES 1.1 | | GLES 2.0 | SYSTEM - |_________| |__________| |__________| LIBRARIES - | | | - - - - | - - - - - - - - - | - - - - - - - - - | - - - - - - | | | - ____v____________________v____________________v____ GUEST - | | KERNEL - | QEMU PIPE | - |___________________________________________________| - | - - - - - - - - - - - - - - -|- - - - - - - - - - - - - - - - - | - v - EMULATOR - - - Specific code inside the emulator program that is capable of transmitting - the wire protocol stream to a special rendering library or process (called - the "renderer" here), which understands the format. - - | - | PROTOCOL BYTE STREAM - _____v_____ - | | - | EMULATOR | - |___________| - | - | UNMODIFIED PROTOCOL BYTE STREAM - _____v_____ - | | - | RENDERER | - |___________| - - - - The renderer decodes the EGL/GLES commands from the wire - protocol stream, and dispatches them to the translator libraries - appropriately. - - | - | PROTOCOL BYTE STREAM - _____v_____ - | | - | RENDERER | - |___________| - | | | - +-----------------+ | +-----------------+ - | | | - ____v____ ___v______ ____v_____ - | | | | | | HOST - |TRANSLATOR |TRANSLATOR| |TRANSLATOR| HOST - | EGL | | GLES 1.1 | | GLES 2.0 | TRANSLATOR - |_________| |__________| |__________| LIBRARIES - - - - - In reality, the protocol stream flows in both directions, even though most - of the commands result in data going from the guest to the host. A complete - picture of the emulation would thus be: - - - - - - _________ __________ __________ - | | | | | | - |EMULATION| |EMULATION | |EMULATION | GUEST - | EGL | | GLES 1.1 | | GLES 2.0 | SYSTEM - |_________| |__________| |__________| LIBRARIES - ^ ^ ^ - | | | - - - - | - - - - - - - - - | - - - - - - - - - | - - - - - - | | | - ____v____________________v____________________v____ GUEST - | | KERNEL - | QEMU PIPE | - |___________________________________________________| - ^ - | - - - - - - - - - - - - - - -|- - - - - - - - - - - - - - - - - | - | PROTOCOL BYTE STREAM - _____v_____ - | | - | EMULATOR | - |___________| - ^ - | UNMODIFIED PROTOCOL BYTE STREAM - _____v_____ - | | - | RENDERER | - |___________| - ^ ^ ^ - | | | - +-----------------+ | +-----------------+ - | | | - ____v____ ___v______ ____v_____ - | | | | | | - |TRANSLATOR |TRANSLATOR| |TRANSLATOR| HOST - | EGL | | GLES 1.1 | | GLES 2.0 | TRANSLATOR - |_________| |__________| |__________| LIBRARIES - ^ ^ ^ - | | | - - - - | - - - - - - - - - | - - - - - - - - - | - - - - - - | | | - ____v____ ____v_____ _____v____ HOST - | | | | | | SYSTEM - | Xgl | | GL 2.0 | | GL 2.0 | LIBRARIES - |_________| |__________| |__________| - - (NOTE: 'Xgl' is for Linux only, replace 'AGL' on OS X, and 'WGL' on Windows). - - -Note that, in the above graphics, only the host system libraries at the bottom -are _not_ provided by Android. - - -Design Requirements: --------------------- - -The above design comes from several important requirements that were decided -early in the project: - -1 - The ability to run the renderer in a separate process from the emulator - itself is important. - - For various practical reasons, we plan to completely separate the core QEMU - emulation from the UI window by using two distinct processes. As such, the - renderer will be implemented as a library inside the UI program, but will - need to receive protocol bytes from the QEMU process. - - The communication channel will be either a fast Unix socket or a Win32 - named pipe between these two. A shared memory segment with appropriate - synchronization primitives might also be used if performance becomes - an issue. - - This explains why the emulator doesn't alter or even try to parse the - protocol byte stream. It only acts as a dumb proxy between the guest - system and the renderer. This also avoids adding lots of GLES-specific - code inside the QEMU code base which is terribly complex. - -2 - The ability to use vendor-specific desktop EGL/GLES libraries is - important. - - GPU vendors like NVidia, AMD or ARM all provide host versions of the - EGL/GLES libraries that emulate their respectivie embedded graphics - chipset. - - The renderer library can be configured to use these instead of the - translator libraries provided with this project. This can be useful to - more accurately emulate the behaviour of specific devices. - - Moreover, these vendor libraries typically expose vendor-specific - extensions that are not provided by the translator libraries. We cannot - expose them without modifying our code, but it's important to be able - to do so without too much pain. - - -Code organization: ------------------- - -All source code for the components above is spread over multiple directories -in the Android source trees: - - - The emulator sources are under $ANDROID/external/qemu, which we'll - call $QEMU in the rest of this document. - - - The guest libraries are under - $ANDROID/development/tools/emulator/opengl, which we'll call $EMUGL_GUEST - - - The host renderer and translator libraries are under - $ANDROID/sdk/emulator/opengl, which we'll call $EMUGL_HOST - - - The QEMU Pipe kernel driver is under $KERNEL/drivers/misc/qemupipe - -Where $ANDROID is the top of the open-source Android source tree, and -$KERNEL is the top of the qemu-specific kernel source tree (using one -of the android-goldfish-xxxx branches here). - -The emulator sources related to this projects are: - - $QEMU/hw/goldfish_pipe.c -> implement QEMU pipe virtual hardware - $QEMU/hw/opengles.c -> implement GLES initialization - $QEMU/hw/hw-pipe-net.c -> implements the communication channel - between the QEMU Pipe and the renderer library - -The other sources are: - - $EMUGL_GUEST/system -> system libraries - $EMUGL_GUEST/shared -> guest copy of shared libraries - $EMUGL_GUEST/tests -> various test programs - $EMUGL_HOST/host -> host libraries (translator + renderer) - $EMUGL_HOST/shared -> host copy of shared libraries - $EMUGL_HOST/tests -> various test programs - -The reason the shared libraries aren't actually shared is historical: at one -point both guest and host code lived in the same place. That turned out to be -impractical with the way the Android SDK is branched, and didn't support the -requirement that a single emulator binary be able to run several generations -of Android. - - -Translator libraries: ---------------------- - -There are three translator host libraries provided by this project: - - libEGL_translator -> EGL 1.2 translation - libGLES_CM_translator -> GLES 1.1 translation - libGLES_V2_translator -> GLES 2.0 translation - -The full name of the library will depend on the host system. -For simplicity, only the library name suffix will change (i.e. the -'lib' prefix is not dropped on Windows), i.e.: - - libEGL_translator.so -> for Linux - libEGL_translator.dylib -> for OS X - libEGL_translator.dll -> for Windows - -The source code for these libraries is located under the following -path in the Android source tree: - - $EMUGL_HOST/host/libs/Translator/EGL - $EMUGL_HOST/host/libs/Translator/GLES_CM - $EMUGL_HOST/host/libs/Translator/GLES_V2 - -The translator libraries also use a common routines defined under: - - $EMUGL_HOST/host/libs/Translator/GLcommon - - -Wire Protocol Overiew: ----------------------- - -The "wire protocol" is implemented as follows: - - - EGL/GLES function calls are described through several "specification" - files, which describes the types, function signatures and various - attributes for each one of them. - - - These files are read by a tool called "emugen" which generates C - source files and headers based on the specification. These correspond - to both encoding, decoding and "wrappers" (more on this later). - - - System "encoder" static libraries are built using some of these generated - files. They contain code that can serialize EGL/GLES calls into simple - byte messages and send it through a generic "IOStream" object. - - - Host "decoder" static libraries are also built using some of these - generated files. Their code retrieves byte messages from an "IOStream" - object, and translates them into function callbacks. - -IOStream abstraction: -- - - - - - - - - - - - -The "IOStream" is a very simple abstract class used to send byte messages -both in the guest and host. It is defined through a shared header under -$EMUGL/host/include/libOpenglRender/IOStream.h - -Note that despite the path, this header is included by *both* host and guest -source code. The main idea around IOStream's design is that to send a message, -one does the following: - - 1/ call stream->allocBuffer(size), which returns the address of a - memory buffer of at least 'size' bytes. - - 2/ write the content of the serialized command (usually a header + some - payload) directly into the buffer - - 3/ call stream->commitBuffer() to send it. - -Alternatively, one can also pack several commands into a single buffer with -stream->alloc() and stream->flush(), as in: - - 1/ buf1 = stream->alloc(size1) - 2/ write first command bytes into buf1 - 3/ buf2 = stream->alloc(size2) - 4/ write second command bytes into buf2 - 5/ stream->flush() - -Finally, there are also explict read/write methods like stream->readFully() -or stream->writeFully() which can be used when you don't want an intermediate -buffer. This is used in certain cases by the implementation, e.g. to avoid -an intermediate memory copy when sending texture data from the guest to the -host. - -The host IOStream implementations are under $EMUGL/shared/OpenglCodecCommon/, -see in particular: - - $EMUGL_HOST/shared/OpenglCodecCommon/TcpStream.cpp - -> using local TCP sockets - $EMUGL_HOST/shared/OpenglCodecCommon/UnixStream.cpp - -> using Unix sockets - $EMUGL_HOST/shared/OpenglCodecCommon/Win32PipeStream.cpp - -> using Win32 named pipes - -The guest IOStream implementation uses the TcpStream.cpp above, as well as -an alternative QEMU-specific source: - - $EMUGL_GUEST/system/OpenglSystemCommon/QemuPipeStream.cpp - -> uses QEMU pipe from the guest - -The QEMU Pipe implementation is _significantly_ faster (about 20x) due to -several reasons: - - - all succesful read() and write() operations through it are instantaneous - from the guest's point of view. - - - all buffer/memory copies are performed directly by the emulator, and thus - much faster than performing the same thing inside the kernel with emulated - ARM instructions. - - - it doesn't need to go through a kernel TCP/IP stack that will wrap the - data into TCP/IP/MAC packets, send them to an emulated ethernet device, - which is itself connected to an internal firewall implementation that - will unwrap the packets, re-assemble them, then send them through BSD - sockets to the host kernel. - -However, would it be necessary, you could write a guest IOStream implementation -that uses a different transport. If you do, please look at -$EMUGL_GUEST/system/OpenglCodecCommon/HostConnection.cpp which contains the -code used to connect the guest to the host, on a per-thread basis. - - -Source code auto-generation: -- - - - - - - - - - - - - - - -The 'emugen' tool is located under $EMUGL_HOST/host/tools/emugen. There is a -README file that explains how it works. - -You can also look at the following specifications files: - -For GLES 1.1: - $EMUGL_HOST/host/GLESv1_dec/gl.types - $EMUGL_HOST/host/GLESv1_dec/gl.in - $EMUGL_HOST/host/GLESv1_dec/gl.attrib - -For GLES 2.0: - $EMUGL_HOST/host/GLESv2_dec/gl2.types - $EMUGL_HOST/host/GLESv2_dec/gl2.in - $EMUGL_HOST/host/GLESv2_dec/gl2.attrib - -For EGL: - $EMUGL_HOST/host/renderControl_dec/renderControl.types - $EMUGL_HOST/host/renderControl_dec/renderControl.in - $EMUGL_HOST/host/renderControl_dec/renderControl.attrib - -Note that the EGL specification files are under a directory named -"renderControl_dec" and have filenames that begin with "renderControl" - -This is mainly for historic reasons now, but is also related to the fact that -this part of the wire protocol contains support functions/calls/specifications -that are not part of the EGL specification itself, but add a few features -required to make everything works. For example, they have calls related to -the "gralloc" system library module used to manage graphics surfaces at a -lower level than EGL. - -Generally speaking, guest encoder sources are located under directories -named $EMUGL_GUEST/system/<name>_enc/, while the corresponding host decoder -sources will be under $EMUGL_HOST/host/libs/<name>_dec/ - -However, all these sources use the same spec files located under the -decoding directories. - -The encoder files are built from emugen and spec files located in $EMUGL_HOST -and copied to the encoder directories in $EMUGL_GUEST by the gen-encoder.sh -script. They are checked in, so that a given version of Android supports a -specific version of the protocol, even if newer versions of the renderer (and -future Android versions) support a newer protocol version. This step needs to -be done manually when the protocol changes; these changes also need to be -accompanied by changes in the renderer to handle the old version of the -protocol. - - -System libraries: ------------------ - -Meta EGL/GLES system libraries, and egl.cfg: -- - - - - - - - - - - - - - - - - - - - - - - -It is important to understand that the emulation-specific EGL/GLES libraries -are not directly linked by applications at runtime. Instead, the system -provides a set of "meta" EGL/GLES libraries that will load the appropriate -hardware-specific libraries on first use. - -More specifically, the system libEGL.so contains a "loader" which will try -to load: - - - hardware-specific EGL/GLES libraries - - the software-based rendering libraries (called "libagl") - -The system libEGL.so is also capable of merging the EGL configs of both the -hardware and software libraries transparently to the application. The system -libGLESv1_CM.so and libGLESv2.so, work with it to ensure that the thread's -current context will be linked to either the hardware or software libraries -depending on the config selected. - -For the record, the loader's source code in under -frameworks/base/opengl/libs/EGL/Loader.cpp. It depends on a file named -/system/lib/egl/egl.cfg which must contain two lines that look like: - - 0 1 <name> - 0 0 android - -The first number in each line is a display number, and must be 0 since the -system's EGL/GLES libraries don't support anything else. - -The second number must be 1 to indicate hardware libraries, and 0 to indicate -a software one. The line corresponding to the hardware library, if any, must -always appear before the one for the software library. - -The third field is a name corresponding to a shared library suffix. It really -means that the corresponding libraries will be named libEGL_<name>.so, -libGLESv1_CM_<name>.so and libGLESv2_<name>.so. Moreover these libraries must -be placed under /system/lib/egl/ - -The name "android" is reserved for the system software renderer. - -The egl.cfg that comes with this project uses the name "emulation" for the -hardware libraries. This means that it provides an egl.cfg file that contains -the following lines: - - 0 1 emulation - 0 0 android - -See $EMUGL_GUEST/system/egl/egl.cfg and more generally the following build -files: - - $EMUGL_GUEST/system/egl/Android.mk - $EMUGL_GUEST/system/GLESv1/Android.mk - $EMUGL_GUEST/system/GLESv2/Android.mk - -to see how the libraries are named and placed under /system/lib/egl/ by the -build system. - - -Emulation libraries: -- - - - - - - - - - - - -The emulator-specific libraries are under the following: - - $EMUGL_GUEST/system/egl/ - $EMUGL_GUEST/system/GLESv1/ - $EMUGL_GUEST/system/GLESv2/ - -The code for GLESv1 and GLESv2 is pretty small, since it mostly link against -the static encoding libraries. - -The code for EGL is a bit more complex, because it needs to deal with -extensions dynamically. I.e. if an extension is not available on the host -it shouldn't be exposed by the library at runtime. So the EGL code queries -the host for the list of available extensions in order to return them to -clients. Similarly, it must query the list of valid EGLConfigs for the -current host system. - - -"gralloc" module implementation: -- - - - - - - - - - - - - - - - - - -In addition to EGL/GLES libraries, the Android system requires a -hardware-specific library to manage graphics surfaces at a level lower than -EGL. This library must be what is called in Android land as a "HAL module". - -A "HAL module" must provide interfaces defined by Android's HAL -(Hardware Abstraction Library). These interface definitions can be found -under $ANDROID/hardware/libhardware/include/ - -Of all possible HAL modules, the "gralloc" one is used by the system's -SurfaceFlinger to allocate framebuffers and other graphics memory regions, -as well as eventually lock/unlock/swap them when needed. - -The code under $EMUGL/system/gralloc/ implements the module required by the -GLES emulation project. It's not very long, but there are a few things to -notice here: - -- first, it will probe the guest system to determine if the emulator that - is running the virtual device really supports GPU emulation. In certain - circumstances this may not be possible. - - If this is the case, then the module will redirect all calls to the - "default" gralloc module that is normally used by the system when - software-only rendering is enabled. - - The probing happens in the function "fallback_init" which gets called - when the module is first opened. This initializes the 'sFallback' variable - to a pointer to the default gralloc module when required. - -- second, this module is used by SurfaceFlinger to display "software surfaces", - i.e. those that are backed by system memory pixel buffers, and written to - directly through the Skia graphics library (i.e. the non-accelerated ones). - - the default module simply copies the pixel data from the surface to the - virtual framebuffer i/o memory, but this project's gralloc module sends it - to the renderer through the QEMU Pipe instead. - - It turns out that this results in _faster_ rendering/frame-rates overall, - because memory copies inside the guest are slow, while QEMU pipe transfers - are done directly in the emulator. - - -Host Renderer: --------------- - -The host renderer library is located under -$EMUGL_HOST/host/libs/libOpenglRender, and it provides an interface described -by the headers under $EMUGL_HOST/host/include/libOpenglRender/render_api.h -(e.g. for use by the emulator). - -In a nutshell, the rendering library is responsible for the following: - - - Providing a virtual off-screen video surface where everything will get - rendered at runtime. Its dimensions are fixed by the call to - initOpenglRender() that must happen just after the library is - initialized. - - - Provide a way to display the virtual video surface on a host application's - UI. This is done by calling createOpenGLSubWindow() which takes as argument - the window ID or handle of a parent window, some display dimensions and - a rotation angle. This allows the surface to be scaled/rotated when it is - displayed, even if the dimensions of the video surface do not change. - - - Provide a way to listen to incoming EGL/GLES commands from the guest. - This is done by providing a so-called "port number" to initOpenglRender(). - - By default, the port number corresponds to a local TCP port number that the - renderer will bind to and listen. Every new connection to this port will - correspond to the creation of a new guest host connection, each such - connection corresponding to a distinct thread in the guest system. - - For performance reasons, it is possible to listen to either Unix sockets - (on Linux and OS X), or to a Win32 named pipe (on Windows). To do so, one - had to call setStreamType() between library initialization - (i.e. initLibrary()) and construction (i.e. initOpenglRender()). - - Note that in these modes, the port number is still used to differentiate - between several emulator instances. These details are normally handled by - the emulator code so you shouldn't care too much. - -Note that an earlier version of the interface allowed a client of the renderer -library to provide its own IOStream implementation. However, this wasn't very -convenient for a number of reasons. This maybe something that could be done -again if it makes sense, but for now the performance numbers are pretty good. - - -Host emulator: --------------- - -The code under $QEMU/android/opengles.c is in charge of dynamically loading -the rendering library and initializing / constructing it properly. - -QEMU pipe connections to the 'opengles' service are piped through the code -in $QEMU/android/hw-pipe-net.c. Look for the openglesPipe_init() function, -which is in charge of creating a connection to the renderer library -(either through a TCP socket, or a Unix pipe depending on configuration. -support for Win32 named pipes hasn't been implemented yet in the emulator) -whenever a guest process opens the "opengles" service through /dev/qemu_pipe. - -There is also some support code for the display of the GLES framebuffer -(through the renderer library's subwindow) under $QEMU/skin/window. - -Note that at the moment, scaling and rotation are supported. However, -brightness emulation (which used to modify the pixel values from the -hardware framebuffer before displaying them) doesn't work. - -Another issue is that it is not possible to display anything on top of the -GL subwindow at the moment. E.g. this will obscure the emulated trackball -image (that is normally toggled with Ctrl-T during emulation, or enabled -by pressing the Delete key). - |