pyprotobuf

protocol buffers compiler.

Converts from .proto to:

  • python protorpc
  • Closure library externs
  • Closure library goog.proto2

Project page: http://code.google.com/p/pyprotobuf

Contents

Supported Languages

ProtoRPC

pyprotobuf can compile proto files to https://code.google.com/p/google-protorpc/.

Example

example.proto:

message Item {
  optional string aString = 1;
  optional int32 aNumber = 2;
  required string aRequiredString = 3;
  repeated string aRepeatedString = 4;
}

Run the command protopy --format python example.proto to generate:

from protorpc import messages

class Item(messages.Message):
    aString = messages.StringField(1)
    aNumber = messages.IntegerField(2)
    aRequiredString = messages.StringField(3, required=True)
    aRepeatedString = messages.StringField(4, repeated=True)
Using the protorpc’s DateTimeField

To use protorpc’s DateTimeField, you must import "protorpc/message_types.proto". This proto file is included with pyprotobuf.

Then you can define fields with the protorpc.DateTimeMessage type.

For example:

import "protorpc/message_types.proto";

message Test {
    optional protorpc.DateTimeMessage datetime = 1;
}

Generates:

from protorpc import messages
import protorpc.message_types


class Test(messages.Message):
    datetime = protorpc.message_types.DateTimeField(1)
Defining custom field types

pyprotobuf supports the option python_field_type to define the protorpc.messages.Message‘s protorpc.messages.Field type.

This can be used to handle (un)serialization of message types to language native types.

For example:

message DateTime {
    required int64 microseconds = 1;
    option python_field_type = "example.module.DateTimeField";
}


message Test {
    optional DateTime datetime = 1;
}

Generates:

from protorpc import messages
import example.module


class DateTime(messages.Message):
    microseconds = messages.IntegerField(1)


class Test(messages.Message):
    datetime = example.module.DateTimeField(1)

In this case, the example.module.DateTimeField` class (not defined) should be customized by converting the microseconds to return a python datetime and vice-versa.

Javascript Externs

Example

Input file (test.proto):

option javascript_package = "com.example";

message Item {
  optional string aString = 1;
  optional int32 aNumber = 2;
  required string aRequiredString = 3;
  repeated string aRepeatedString = 4;
}

Generated javascript externs(protopy --format externs test.proto):

/** @constructor */
com.example.Item = function(){};

/** @type {string} */
com.example.Item.prototype.aString;

/** @type {number} */
com.example.Item.prototype.aNumber;

/** @type {string} */
com.example.Item.prototype.aRequiredString;

/** @type {[string]} */
com.example.Item.prototype.aRepeatedString;

Javascript Closure library

Golang

Extending pyprotobuf

Developing a custom generator

Create a setup.py

Custom generators can be registered by creating a setup.py with the entry point pyprotobuf.generators. pyprotobuf will detect the generator and make it available as a format.

setup.py:

setup(
    # ...
    entry_points = '''
    [pyprotobuf.generators]
    custom = custom_generator
    '''
    # ...
 )
Create a generator module

Note

The CodeGenerator api is unstable.

custom_generator.py:

from pyprotobuf.codegenerator import CodeGenerator

class Generator(CodeGenerator):
    def generate_file(self, protonode, **kwargs):
        """ Custom ProtoNode generating logic

            :return: The compiled code
            :rtype: str
        """
        pass

__generator__ = Generator
The compiler AST

The compiler produces a tree in the form of:

  • RootNode * PackageNode

    • FileNode * MessageNode * ...

FileNodes without packages declarations are placed in an unnamed PackageNode under the root.

API Documentation

Compiler

class pyprotobuf.compiler.Compiler[source]

Nodes

class pyprotobuf.nodes.ParseNode[source]
children = None
Type:list[pyprotobuf.nodes.ParseNode]
parent = None
name = None
comment = None

A comment associated with this node.

Type:pyprotobuf.nodes.CommentNode
get_file()[source]
add_child(c)[source]
get_children()[source]
get_child(index)[source]
get_full_typename()[source]
add_dependency(dep)[source]
Parameters:dep (pyprotobuf.nodes.ParseNode) – The node to add a dependency to.
get_dependencies()[source]
Return type:list[pyprotobuf.nodes.ParseNode]
get_parents()[source]
Return type:list[pyprotobuf.nodes.ParseNode]
get_root()[source]
get_full_name()[source]
get_children_of_type(node_class)[source]
Return type:list[V <= T]
resolve_name(name)[source]
has_option(name)[source]
set_option(name, value)[source]
get_option(*args)[source]
to_proto()[source]
exception pyprotobuf.nodes.InvalidChildTypeError(child, accepted_types)[source]
class pyprotobuf.nodes.RootNode[source]
get_package(name)[source]
add_child(c)[source]
class pyprotobuf.nodes.FileNode[source]
filename = None
package_name = None
get_imports()[source]
class pyprotobuf.nodes.CommentNode[source]
class pyprotobuf.nodes.Package[source]
name = ''
add_child(c)[source]
is_named()[source]
class pyprotobuf.nodes.PackageDefinition[source]
name = None
to_proto()[source]
class pyprotobuf.nodes.ServiceNode[source]
name = None
to_proto()[source]
class pyprotobuf.nodes.MethodNode[source]
name = None
request_type = None
response_type = None
class pyprotobuf.nodes.MessageNode[source]
name = None
tostr(depth)[source]
to_proto()[source]
class pyprotobuf.nodes.FieldDescriptorNode[source]
Properties:
label: One of “repeated”, “optional” or “required”. number: The tag/number/id of the field. name: The name of the field. type: Can be a string (Enum of proto types), MessageNode or EnumNode.
label = None
number = None
name = None
type = None
class LabelType[source]
REPEATED = 'repeated'
OPTIONAL = 'optional'
REQUIRED = 'required'
FieldDescriptorNode.to_proto()[source]
class pyprotobuf.nodes.EnumNode[source]
name = None
has(key)[source]
get(key)[source]
to_proto()[source]
class pyprotobuf.nodes.EnumAssignmentNode[source]
name = None
value = None
to_proto()[source]
class pyprotobuf.nodes.OptionNode[source]
name = None
value = None
to_proto()[source]
class pyprotobuf.nodes.ExtendNode[source]
name = None
message_node[source]
Return type:MessageNode
to_proto()[source]
class pyprotobuf.nodes.OptionalNode[source]
type = None
to_proto()[source]
class pyprotobuf.nodes.SyntaxNode[source]
to_proto()[source]
class pyprotobuf.nodes.ImportNode[source]

value: path

value = None
file_node = None
public = False
to_proto()[source]
class pyprotobuf.nodes.ExtensionsNode[source]
to_proto()[source]
class pyprotobuf.nodes.Types[source]
BOOL = 'bool'
STRING = 'string'
INT32 = 'int32'
INT64 = 'int64'
UINT32 = 'uint32'
UINT64 = 'uint64'
SINT32 = 'sint32'
SINT64 = 'sint64'
FIXED32 = 'fixed32'
FIXED64 = 'fixed64'
SFIXED32 = 'sfixed32'
SFIXED64 = 'sfixed64'
DOUBLE = 'double'
FLOAT = 'float'
BYTES = 'bytes'
ENUM = 'enum'
MESSAGE = 'messsage'
GROUP = 'group'

Indices and tables