Deep Dive into Android IPC/Binder Framework(1/2)

URL : https://thenewcircle.com/s/post/1340/Deep_Dive_Into_Binder_Presentation.htm

Objectives

- Binder Overview

- IPC

- Advantages of Binder

- Binder vs Intent/ContentProvider/Messenger-based IPC

- Binder Terminology

- Binder Communication and Discovery

- AIDL

- Binder Object Reference Mapping

- Binder by Example

- Async Binder

- Memory Sharing

- Binder Limitations

- Security

.....

What is Binder?

- An IPC/component system for developing object-oriented OS services

   + Not yet another object-oriented kernel

   + Instead an object-oriented operating system environment that works on traditional kernels, like Linux!

- Essential to Android!

- Comes from OpenBinder

   + Started at Be, Inc. as a key part of the "next generation BeOS" (~ 2001)

   + Acquired by PalmSource

   + First implementation used in Palm Cobalt (micro-kernel based OS)

   + Palm switched to Linux, so Binder ported to Linux, open-sourced (~ 2005)

   + Google hired Dianne Hackborn, a key OpenBinder engineer, to join the Android team

   + Used as-is for the initial bring-up of Android, but then completely rewritten (~ 2008)

   + OpenBinder no longer maintained - long live Binder!

- Focused on scalability, stability, flexibility, low-latency/overhead, easy programming model

IPC

- Inter-process communication (IPC) is a framework for the exchange of signals and data across multiple processes

- Used for message passing, synchronization, shared memory, and remote procedure calls (RPC)

- Enables information sharing, computational speedup, modularity, convenience, privilege separation, data isolation, stability

    + Each process has its own (sandboxed) address space, typically running under a unique system ID

- Many IPC options

    + Files (including memory mapped)

    + Signals

    + Sockets (UNIX domain, TCP/IP)

    + Pipes (including named pipes)

    + Semaphores

    + Shared memory

    + Message passing (including queues, message bus)

    + Intents, ContentProviders, Messenger

    + Binder!

Why Binder?

- Android apps and system services run in separate processes for security, stability, and memory management reasons, but they need to communicate and share data!

    + Security: each process is sandboxed and run under a distinct system identity

    + Stability: if a process misbehaves (e.g. crashes), it does not affect any other processes

    + Memory management: "unneeded" processes are removed to free resources (mainly memory) for new ones

    + In fact, a single Android app can have its components run in separate processes

- IPC to the rescue

    + But we need to avoid overhead of traditional IPC and avoid denial of service issues

- Android’s libc (a.k.a. bionic) does not support System V IPCs,

    + No SysV semaphores, shared memory segments, message queues, etc.

    + System V IPC is prone to kernel resource leakage, when a process "forgets" to release shared IPC resources upon termination

    + Buggy, malicious code, or a well-behaved app that is low-memory SIGKILL'ed

- Binder to the rescue!

    + Its built-in reference-counting of "object" references plus death-notification mechanism make it suitable for "hostile" environments (where lowmemorykiller roams)

    + When a binder service is no longer referenced by any clients, its owner is automatically notified that it can dispose of it

- Many other features:

    + "Thread migration" - like programming model:

         * Automatic management of thread-pools

         * Methods on remote objects can be invoked as if they were local - the thread appears to "jump" to the other process

         * Synchronous and asynchronous (oneway) invocation model

    + Identifying senders to receivers (via UID/PID) - important for security reasons

    + Unique object-mapping across process boundaries

         * A reference to a remote object can be passed to yet another process and can be used as an identifying token

    + Ability to send file descriptors across process boundaries

    + Simple Android Interface Definition Language (AIDL)

    + Built-in support for marshalling many common data-types

    + Simplified transaction invocation model via auto-generated proxies and stubs (Java-only)

    + Recursion across processes - i.e. behaves the same as recursion semantics when calling methods on local objects

    + Local execution mode (no IPC/data marshalling) if the client and the service happen to be in the same process

- But:

    + No support for RPC (local-only)

    + Client-service message-based communication - not well-suited for streaming

    + Not defined by POSIX or any other standard

- Most apps and core system services depend on Binder

    + Most app component life-cycle call-backs (e.g. onResume(), onDestory(), etc.) are invoked by ActivityManagerService via binder

    + Turn off binder, and the entire system grinds to a halt (no display, no audio, no input, no sensors, …)

    + Unix domain sockets used in some cases (e.g. RILD)

IPC with Intents and ContentProviders?

- Android supports a simple form of IPC via intents and content providers

- Intent messaging is a framework for asynchronous communication among Android components

    + Those components may run in the same or across different apps (i.e. processes)

    + Enables both point-to-point as well as publish-subscribe messaging domains

    + The intent itself represents a message containing the description of the operation to be performed as well as data to be passed to the recipient(s)

    + Implicit intents enable loosely-coupled APIs

- ContentResolvers communicate synchronously with ContentProviders (typically running in separate apps) via a fixed (CRUD) API

- All android component can act as a senders, and most as receivers

- All communication happens on the Looper (a.k.a. main) thread (by default)

- But:

    + Not really OOP

    + Asynchronous-only model for intent-based communication

    + Not well-suited for low-latency

    + Since the APIs are loosely-defined, prone to run-time errors

    + All underlying communication is based on Binder!

    + In fact, Intents and ContentProvider are just a higher-level abstraction of Binder

    + Facilitated via system services: ActivityManagerService and PackageManagerService

Messenger IPC

- Android’s Messenger represents a reference to a Handler that can be sent to a remote process via an Intent

- A reference to the Messenger can be sent via an Intent using the previously mentioned IPC mechanism

- Messages sent by the remote process via the messenger are delivered to the local handler

- Messages are like Intents, in that they can designate the "operation" (aMessage.what) and data(aMessage.getData())

- Still asynchronous, but lower latency/overhead

- Great for efficient call-backs from the service to the client

- Messages are by default handled on the Looper thread

- All underlying communication is still based on Binder!

Binder Terminology

- Binder (Framework)

: The overall IPC architecture

- Binder Driver

: The kernel-level driver that fascinates the communication across process boundaries

- Binder Protocol

: Low-level protocol (ioctl-based) used to communicate with the Binder driver

- IBinder Interface

: A well-defined behavior (i.e. methods) that Binder Objects must implement

- AIDL

: Android Interface Definition Language used to describe business operations on an IBinder Interface

- Binder (Object)

: A generic implementation of the IBinder interface

- Binder Token

: An abstract 32-bit integer value that uniquely identifies a Binder object across all processes on the system

- Binder Service

: An actual implementation of the Binder (Object) that implements the business operations

- Binder Client

: An object wanting to make use of the behavior offered by a binder service

- Binder Transaction

: An act of invoking an operation (i.e. a method) on a remote Binder object, which may involve sending/receiving data, over the Binder Protocol

- Parcel

: "Container for a message (data and object references) that can be sent through an IBinder." A unit of transactional data - one for the outbound request, and another for the inbound reply

- Marshalling

: A procedure for converting higher level applications data structures (i.e. request/response parameters) into parcels for the purposes of embedding them into Binder transactions

- Unmarshalling

: A procedure for reconstructing higher-level application data-structures (i.e. request/response parameters) from parcels received through Binder transactions

- Proxy

: An implementation of the AIDL interface that un/marshals data and maps method calls to transactions submitted via a wrapped IBinder reference to the Binder object

- Stub

: A partial implementation of the AIDL interface that maps transactions to Binder Service method calls while un/marshalling data

- Context Manager (a.k.a. servicemanager)

: A special Binder Object with a known handle (registered as handle 0) that is used as a registry/lookup service for other Binder Objects (name → handle mapping)

Binder Communication and Discovery

- As far as the client is concerned, it just wants to use the service:

- While processes cannot directly invoke operations (or read/write data) on other processes, the kernel can, so they make use of the Binder driver:

* Since the service may get concurrent requests from multiple clients, it needs to protect (synchronize access to) its mutable state.

- Binder driver is exposed via /dev/binder and offers a relatively simple API based on open, release, poll, mmap, flush, and ioctl operations.

- In fact most communication happens via ioctl(binderFd, BINDER_WRITE_READ, &bwd), where bwd is defined as:

struct binder_write_read {
  signed long write_size; /* bytes to write */
  signed long write_consumed; /* bytes consumed by driver */
  unsigned long write_buffer;
  signed long read_size;  /* bytes to read */
  signed long read_consumed;  /* bytes consumed by driver */
  unsigned long read_buffer;
};

- The write_buffer contains a series of commands for the driver to perform

    + Book-keeping commands, e.g. inc/decrement binder object references, request/clear death notification, etc.

    + A command requiring a response, like BC_TRANSACTION

- Upon returning, the read_buffer will contain commands for the user-space to perform

    + Same book-keeping commands

    + A command requesting processing of the response (i.e. BC_REPLY) or a request to perform a nested (recursive) operation

- Clients communicate with services via transactions, which contain a binder token, code of the method to execute, raw data buffer, and sender PID/UID (added by the driver)

- Most-low-level operations and data structures (i.e. Parcel) are abstracted by libbinder (at the native level), which is what the clients and services use.

- Except that clients and services don’t want to know anything about the Binder protocol and libbinder, so they make use of proxies and stubs:

# Java-based proxies and stubs can be automatically generated by aidl tool for services described with AIDL.

- In fact, most clients don’t even want to know that they are using IPC, never mind Binder, or proxies, so they count on managers to abstract all of that complexity for them:

# This is in particular true for system services, which typically expose only a subset of their APIs to the clients via their managers.

- But how does the client get a handle to the service it wants to talk to? Just ask the servicemanager (Binder’s CONTEXT_MGR), and hope that the service has already registered with it:

# For security/sanity reasons, the binder driver will only accept a single/one-time CONTEXT_MGR registration, which is why servicemanager is among the first services to start on Android.

! Tip

To get a list of services currently registered with servicemanager, run:

$ adb shell service list
Found 71 services:
0 sip: [android.net.sip.ISipService]
1 phone: [com.android.internal.telephony.ITelephony]
…
20  location: [android.location.ILocationManager]
…
55  activity: [android.app.IActivityManager]
56  package: [android.content.pm.IPackageManager]
…
67  SurfaceFlinger: [android.ui.ISurfaceComposer]
68  media.camera: [android.hardware.ICameraService]
69  media.player: [android.media.IMediaPlayerService]
70  media.audio_flinger: [android.media.IAudioFlinger]

Another way to look at it:

 Location Service: An Example

.....

Binder Object Reference Mapping Across Process Boundaries

- A binder object reference is one of the following

    + An actual virtual memory address to a binder object in the same process

    + An abstract 32-bit handle to a binder object in another process

- On every transaction, the binder driver automatically maps local addresses to remote binder handles and remote binder handles to local addresses

- This mapping is done on:

    + Targets of binder transactions

    + IBinder object references shared across process boundaries as a parameter or a return value (embedded in transaction data)

- For this to work

    + The driver maintains mappings of local addresses and remote handles between processes (as a binary-tree per process) so that it can perform this translation

    + References embedded in transaction data are discovered based on offsets that the client provides when it submits its transaction and then rewritten in-place

- The binder driver does not know anything about binder objects that have never been shared with a remote process

    + Once a new binder object reference is found in a transaction, it is remembered by binder

    + Any time that reference is shared with another process, its reference count is incremented

    + Reference count is decremented either explicitly or automatically, when the process dies

    + When a reference is no longer needed, its owner is notified that it can be released, and binder removes its mapping

Building a Binder-based Service and Client by Example

- To demonstrate a Binder-based service and client (based on Fibonacci), we’ll create three separate projects:

    1. FibonacciCommon library project - to define our AIDL interface as well as custom types for parameters and return values

    2. FibonacciService project - where we implement our AIDL interface and expose it to the clients

    3. FibonacciClient project - where we connect to our AIDL-defined service and use it

- The code is available

As a ZIP archive: https://github.com/marakana/FibonacciBinderDemo/zipball/master

By Git: git clone https://github.com/marakana/FibonacciBinderDemo.git