Welcome to Hyperledger fabric’s documentation¶

Blockchain Network¶

A blockchain network consists of, at minimum, one peer (responsible for endorsing and committing transactions) leveraging an ordering service, and a membership services component (certificate authority) that distributes and revokes cryptographic certificates representative of user identities and permissions.

Permissioned Network¶

A blockchain network where any entity (node) is required to maintain a member identity on the network. End users must be authorized and authenticated in order to use the network.

Peer¶

Peer is a component that executes, and maintains a ledger of, transactions.   There are two roles for a peer – endorser and committer.  The architecture has been designed such that a peer is always a committer, but not necessarily always an endorser. Peers play no role in the ordering of transactions.

Member¶

A Member is a participant (such as a company or organization) that operates components - Peers, Orderers, and applications - in the blockchain network.  A member is identified by its CA certificate (i.e. a unique enrollment).   A Member’s peer will be leveraged by end users in order to perform transaction operations on specific channels.

Transaction¶

Refers to an operation in which an authorized end user performs read/write operations against the ledger. There are three unique types of transactions - deploy, invoke, and query.

End User¶

An end user is someone who would interact with the blockchain through a set of published APIs (i.e. the hfc SDK).  You can have an admin user who will typically grant permissions to the Member’s components, and a client user, who, upon proper authentication through the admin user, will drive chaincode applications (deploy, invoke, query) on various channels.  In the case of self-executing transactions, the application itself can also be thought of as the end user.

Ordering Service¶

A centralized or decentralized service that orders transactions in a block.  You can select different implementations of the “ordering” function - e.g “solo” for simplicity and testing, Kafka for crash fault tolerance, or sBFT/PBFT for byzantine fault tolerance. You can also develop your own protocol to plug into the service.

Consensus¶

A broader term overarching the entire transactional flow, which serves to generate an agreement on the order and to confirm the correctness of the set of transactions constituting a block.

Orderer¶

One of the network entities that form the ordering service. A collection of ordering service nodes (OSNs) will order transactions into blocks according to the network’s chosen ordering implementation. In the case of “solo”, only one OSN is required. Transactions are “broadcast” to orderers, and then “delivered” as blocks to the appropriate channel.

Endorser¶

A specific peer role, where the Endorser peer is responsible for simulating transactions, and in turn preventing unstable or non-deterministic transactions from passing through the network.  A transaction is sent to an endorser in the form of a transaction proposal.  All endorsing peers are also committing peers (i.e. they write to the ledger).

Committer¶

A specific peer role, where the Committing peer appends the validated transactions to the channel-specific ledger.  A peer can act as both an endorser and committer, but in more regulated circumstances might only serve as a committer.

Bootstrap¶

The initial setup of a network.  There is the bootstrap of a peer network, during which policies, system chaincodes, and cryptographic materials (certs) are disseminated amongst participants, and the bootstrap of an ordering network.  The bootstrap of the ordering network must precede the bootstrap of the peer network, as a peer network is contingent upon the presence of an ordering service. A network need only be “bootstrapped” once.

Block¶

A batch of ordered transactions, potentially containing ones of an invalid nature, that is delivered to the peers for validation and committal.

System chain¶

The system chain can be thought of as the common binding for a channel or group of channels. For instance, a collection of financial institutions may form a consortium (represented through the system chain), and then proceed to create channels relative to their aligned and varying business agendas.

Channel¶

A Channel is formed as an offshoot of the system chain; and best thought of as a “topic” for peers to subscribe to, or rather, a subset of a broader blockchain network. A peer may subscribe on various channels and can only access the transactions on the subscribed channels.  Each channel will have a unique ledger, thus accommodating confidentiality and execution of multilateral contracts.

Multi-channel¶

The fabric will allow for multiple channels with a designated ledger per channel.  This capability allows for multilateral contracts where only the restricted participants on the channel will submit, endorse, order, or commit transactions on that channel.  As such, a single peer can maintain multiple ledgers without compromising privacy and confidentiality.

Configuration Block¶

Contains the configuration data defining members and policies for a system chain or channel(s). Any changes to the channel(s) or overall network (e.g. a new member successfully joining) will result in a new configuration block being appended to the appropriate chain.  This block will contain the contents of the genesis block, plus the delta. The policy to alter or edit a channel-level configuration block is defined through the Configuration System Chaincode (CSCC).

Genesis Block¶

The configuration block that initializes a blockchain network or channel, and also serves as the first block on a chain.

Ledger¶

An append-only transaction log managed by peers.  Ledger keeps the log of ordered transaction batches. There are two denotations for ledger; peer and validated. The peer ledger contains all batched transactions coming out of the ordering service, some of which may in fact be invalid. The validated ledger will contain fully endorsed and validated transaction blocks. In other words, transactions in the validated ledger have passed the entire gamut of “consensus” - i.e. they have been endorsed, ordered, and validated.

Dynamic membership¶

he fabric will allow for endorsers and committers to come and go based on membership, and the blockchain network will continue to operate. Dynamic membership is critical when businesses grow and members need to be added or removed for various reasons.

Query/Non-Key Value Query¶

using couchDB 2.0 you now have the capability to leverage an API to perform more complex queries against combinations of variables, including time ranges, transaction types, users, etc.  This feature allows for auditors and regulators to aggregate and mine large chunks of data.

Gossip Protocol¶

A communication protocol used among peers in a channel, to maintain their network and to elect Leaders, through which funnels all communications with the Ordering Service. Gossip allows for data dissemination, therein providing support for scalability due to the fact that not all peers are required to execute transactions and communicate with the ordering service.

System Chaincode¶

System Chaincode (SCC) is a chaincode built with the peer and run in the same process as the peer. SCC is responsible for broader configurations of fabric behavior, such as timing and naming services.

Lifecycle System Chaincode¶

Lifecycle System Chaincode (LSCC) is a system chaincode that handles deployment, upgrade and termination transactions for user chaincodes.

Configuration System Chaincode¶

Configuration System Chaincode (CSCC) is a “management” system chaincode that handles configuration requests to alter an aspect of a channel (e.g. add a new member).  The CSCC will interrogate the channel’s policies to determine if a new configuration block can be created.

Endorsement System Chaincode¶

Endorsement System Chaincode (ESCC) is a system chaincode that andles the endorsement policy for specific pieces of chaincode deployed on a network, and defines the necessary parameters (percentage or combination of signatures from endorsing peers) for a transaction proposal to receive a successful proposal response (i.e. endorsement).  Deployments and invocations of user chaincodes both require a corresponding ESCC, which is defined at the time of the deployment transaction proposal for the user chaincode.

Validation System Chaincode¶

Validation System Chaincode (VSCC) Handles the validation policy for specific pieces of chaincode deployed on a network.  Deployments and invocations of user chaincodes both require a corresponding VSCC, which is defined at the time of the deployment transaction proposal for the user chaincode. VSCC validates the specified level of “endorsement” (i.e. endorsement policy) in order to prevent malicious or faulty behavior from the client.

Policy¶

There are policies for endorsement, validation, block committal, chaincode management and network/channel management.  Policies are defined through system chaincodes, and contain the requisite specifications for a network action to succeed.  For example, an endorsement policy may require that 100% of endorsers achieve the same result upon transaction simulation.

Endorsement policy¶

A blockchain network must establish rules that govern the endorsement (or not) of proposed, simulated transactions. This endorsement policy could require that a transaction be endorsed by a minimum number of endorsing peers, a minimum percentage of endorsing peers, or by all endorsing peers that are assigned to a specific chaincode application. Policies can be curated based on the application and the desired level of resilience against misbehavior (deliberate or not) by the endorsing peers. A distinct endorsement policy for deploy transactions, which install new chaincode, is also required.

Proposal¶

A transaction request sent from a client or admin user to one or more peers in a network; examples include deploy, invoke, query, or configuration request.

Deploy¶

Refers to the function through which chaincode applications are deployed on chain. A deploy is first sent from the client SDK or CLI to a Lifecycle System Chaincode in the form of a proposal.

Invoke¶

Used to call chaincode functions.  Invocations are captured as transaction proposals, which then pass through a modular flow of endorsement, ordering, validation, committal.  The structure of invoke is a function and an array of arguments.

Membership Services¶

Membership Services manages user identities on a permissioned blockchain network; this function is implemented through the fabric-ca component.  fabric-ca is comprised of a client and server, and handles the distribution and revocation of enrollment materials (certificates), which serve to identify and authenticate users on a network.

The in-line MembershipSrvc code (MSP) runs on the peers themselves, and is used by the peer when authenticating transaction processing results, and by the client to verify/authenticate transactions. Membership Services provides a distinction of roles by combining elements of Public Key Infrastructure (PKI) and decentralization (consensus). By contrast, non-permissioned networks do not provide member-specific authority or a distinction of roles.

A permissioned blockchain requires entities to register for long-term identity credentials (Enrollment Certificates), which can be distinguished according to entity type. For users, an Enrollment Certificate authorizes the Transaction Certificate Authority (TCA) to issue pseudonymous credentials; these certificates authorize transactions submitted by the user. Transaction certificates persist on the blockchain, and enable authorized auditors to associate, and identify the transacting parties for otherwise un-linkable transactions.

Membership Service Provider¶

The Membership Service Provider (MSP) refers to an abstract component of the system that provides (anonymous) credentials to clients, and peers for them to participate in a Hyperledger/fabric network. Clients use these credentials to authenticate their transactions, and peers use these credentials to authenticate transaction processing results (endorsements). While strongly connected to the transaction processing components of the systems, this interface aims to have membership services components defined, in such a way that alternate implementations of this can be smoothly plugged in without modifying the core of transaction processing components of the system.

Initialize¶

A chaincode method to define the assets and parameters in a piece of chaincode prior to issuing deploys and invocations.  As the name implies, this function should be used to do any initialization to the chaincode, such as configure the initial state of a key/value pair on the ledger.

appshim¶

An application client used by ordering service nodes to process “broadcast” messages arriving from clients or peers. This shim allows the ordering service to perform membership-related functionality checks. In other words, is a peer or client properly authorized to perform the requested function (e.g. upgrade chaincode or reconfigure channel settings).

osshim¶

An ordering service client used by the application to process ordering service messages (i.e. “deliver” messages) that are advertised within a channel.

Hyperledger Fabric Client SDK¶

Provides a powerful set of APIs and contains myriad “methods” or “calls” that expose the capabilities and functionalities in the Hyperledger Fabric code base.  For example, addMember, removeMember. The Fabric SDK comes in multiple flavors - Node.js, Java, and Python, for starters - thus, allowing developers to write application code in any of those programming languages.

Chaincode¶

Embedded logic that encodes the rules for specific types of network transactions. Developers write chaincode applications, which are then deployed onto a chain by an appropriately authorized member. End users then invoke chaincode through a client-side application that interfaces with a network peer. Chaincode runs network transactions, which if validated, are appended to the shared ledger and modify world state.

Prerequisites and setup¶

• Docker - v1.13 or higher

• Docker Compose - v1.8 or higher

• Node.js & npm - node v6.9.5 and npm v3.10.10 If you already have node on your machine, use the node website to install v6.9.5 or issue the following command in your terminal:

  nvm install v6.9.5
  

  then execute the following to see your versions:

  # should be 6.9.5
  node -v
  

  AND

  # should be 3.10.10
  npm -v
  

Curl the source code to create network entities¶

• Download the cURL tool if not already installed.

• Determine a location on your local machine where you want to place the Fabric artifacts and application code.

  mkdir -p <my_dev_workspace>/hackfest
  cd <my_dev_workspace>/hackfest
  

  Next, execute the following command:

  curl -L https://raw.githubusercontent.com/hyperledger/fabric/master/examples/sfhackfest/sfhackfest.tar.gz -o sfhackfest.tar.gz 2> /dev/null;  tar -xvf sfhackfest.tar.gz
  

  This command pulls and extracts all of the necessary artifacts to set up your network - Docker Compose script, channel generate/join script, crypto material for identity attestation, etc. In the /src/github.com/example_cc directory you will find the chaincode that will be deployed.

Your directory should contain the following:

JDoe-mbp: JohnDoe$ pwd
/Users/JohnDoe
JDoe-mbp: JohnDoe$ ls
sfhackfest.tar.gz   channel_test.sh   src
ccenv     docker-compose-gettingstarted.yml  tmp

Using Docker¶

You do not need to manually pull any images. The images for - fabric-peer, fabric-orderer, fabric-ca, and cli are specified in the .yml file and will automatically download, extract, and run when you execute the docker-compose command.

Commands¶

The channel commands are:

• create - create and name a channel in the orderer and get back a genesis block for the channel. The genesis block is named in accordance with the channel name.
• join - use the genesis block from the create command to issue a join request to a peer.

Use Docker to spawn network entities & create/join a channel¶

Ensure the hyperledger/fabric-ccenv image is tagged as latest:

docker-compose -f docker-compose-gettingstarted.yml build

Create network entities, create channel, join peers to channel:

docker-compose -f docker-compose-gettingstarted.yml up -d

Behind the scenes this started six containers (3 peers, a “solo” orderer, cli and CA) in detached mode. A script - channel_test.sh - embedded within the docker-compose-gettingstarted.yml issued the create channel and join channel commands within the CLI container. In the end, you are left with a network and a channel containing three peers - peer0, peer1, peer2.

View your containers:

# if you have no other containers running, you will see six
docker ps

Ensure the channel has been created and peers have successfully joined:

docker exec -it cli bash

You should see the following in your terminal:

/opt/gopath/src/github.com/hyperledger/fabric/peer #

To view results for channel creation/join:

more results.txt

You’re looking for:

SUCCESSFUL CHANNEL CREATION
SUCCESSFUL JOIN CHANNEL on PEER0
SUCCESSFUL JOIN CHANNEL on PEER1
SUCCESSFUL JOIN CHANNEL on PEER2

To view genesis block:

more myc1.block

Exit the cli container:

exit

Curl the application source code and SDK modules¶

• Prior to issuing the command, make sure you are in the same working directory where you curled the network code. AND make sure you have exited the cli container.

• Execute the following command:

  curl -OOOOOO https://raw.githubusercontent.com/hyperledger/fabric-sdk-node/v1.0-alpha/examples/balance-transfer/{config.json,deploy.js,helper.js,invoke.js,query.js,package.json}
  

This command pulls the javascript code for issuing your deploy, invoke and query calls. It also retrieves dependencies for the node SDK modules.

• Install the node modules:

  # You may be prompted for your root password at one or more times during this process.
  npm install
  

  You now have all of the necessary prerequisites and Fabric artifacts.

Use node SDK to register/enroll user and deploy/invoke/query¶

The individual javascript programs will exercise the SDK APIs to register and enroll the client with the provisioned Certificate Authority. Once the client is properly authenticated, the programs will demonstrate basic chaincode functionalities - deploy, invoke, and query. Make sure you are in the working directory where you pulled the source code before proceeding.

Upon success of each node program, you will receive a “200” response in the terminal.

Register/enroll & deploy chaincode (Linux or OSX):

# Deploy initializes key value pairs of "a","100" & "b","200".
GOPATH=$PWD node deploy.js

Register/enroll & deploy chaincode (Windows):

# Deploy initializes key value pairs of "a","100" & "b","200".
SET GOPATH=%cd%
node deploy.js

Issue an invoke. Move units 100 from “a” to “b”:

node invoke.js

Query against key value “b”:

# this should return a value of 300
node query.js

Explore the various node.js programs, along with example_cc.go to better understand the SDK and APIs.

Manually create and join peers to a new channel¶

Use the cli container to manually exercise the create channel and join channel APIs.

Channel - myc1 already exists, so let’s create a new channel named myc2.

Exec into the cli container:

docker exec -it cli bash

If successful, you should see the following in your terminal:

/opt/gopath/src/github.com/hyperledger/fabric/peer #

Send createChannel API to Ordering Service:

CORE_PEER_COMMITTER_LEDGER_ORDERER=orderer:7050 peer channel create -c myc2

This will return a genesis block - myc2.block - that you can issue join commands with. Next, send a joinChannel API to peer0 and pass in the genesis block as an argument. The channel is defined within the genesis block:

CORE_PEER_COMMITTER_LEDGER_ORDERER=orderer:7050 CORE_PEER_ADDRESS=peer0:7051 peer channel join -b myc2.block

To join the other peers to the channel, simply reissue the above command with peer1 or peer2 specified. For example:

CORE_PEER_COMMITTER_LEDGER_ORDERER=orderer:7050 CORE_PEER_ADDRESS=peer1:7051 peer channel join -b myc2.block

Once the peers have all joined the channel, you are able to issues queries against any peer without having to deploy chaincode to each of them.

Use cli to deploy, invoke and query¶

Run the deploy command. This command is deploying a chaincode named mycc to peer0 on the Channel ID myc2. The constructor message is initializing a and b with values of 100 and 200 respectively.

CORE_PEER_ADDRESS=peer0:7051 CORE_PEER_COMMITTER_LEDGER_ORDERER=orderer:7050 peer chaincode deploy -C myc2 -n mycc -p github.com/hyperledger/fabric/examples -c '{"Args":["init","a","100","b","200"]}'

Run the invoke command. This invocation is moving 10 units from a to b.

CORE_PEER_ADDRESS=peer0:7051 CORE_PEER_COMMITTER_LEDGER_ORDERER=orderer:7050 peer chaincode invoke -C myc2 -n mycc -c '{"function":"invoke","Args":["move","a","b","10"]}'

Run the query command. The invocation transferred 10 units from a to b, therefore a query against a should return the value 90.

CORE_PEER_ADDRESS=peer0:7051 CORE_PEER_COMMITTER_LEDGER_ORDERER=orderer:7050 peer chaincode query -C myc2 -n mycc -c '{"function":"invoke","Args":["query","a"]}'

You can issue an exit command at any time to exit the cli container.

Creating your initial channel through the cli¶

If you want to manually create the initial channel through the cli container, you will need to edit the Docker Compose file. Use an editor to open docker-compose-gettingstarted.yml and comment out the channel_test.sh command in your cli image. Simply place a # to the left of the command. (Recall that this script is executing the create and join channel APIs when you run docker-compose up) For example:

cli:
  container_name: cli
  <CONTENT REMOVED FOR BREVITY>
  working_dir: /opt/gopath/src/github.com/hyperledger/fabric/peer
#  command: sh -c './channel_test.sh; sleep 1000'
#  command: /bin/sh

Then use the cli commands from the prior two sections.

Troubleshooting (optional)¶

If you have existing containers running, you may receive an error indicating that a port is already occupied. If this occurs, you will need to kill the container that is using said port.

If a file cannot be located, make sure your curl commands executed successfully and make sure you are in the directory where you pulled the source code.

If you are receiving timeout or GRPC communication errors, make sure you have the correct version of Docker installed - v1.13.0. Then try restarting your failing docker process. For example:

docker stop peer0

Then:

docker start peer0

Another approach to GRPC and DNS errors (peer failing to resolve with orderer and vice versa) is to hardcode the IP addresses for each. You will know if there is a DNS issue, because a more results.txt command within the cli container will display something similar to:

ERROR CREATING CHANNEL
PEER0 ERROR JOINING CHANNEL

Issue a docker inspect <container_name> to ascertain the IP address. For example:

docker inspect peer0 | grep IPAddress

AND

docker inspect orderer | grep IPAddress

Take these values and hard code them into your cli commands. For example:

CORE_PEER_COMMITTER_LEDGER_ORDERER=172.21.0.2:7050 peer channel create -c myc1

AND THEN

CORE_PEER_COMMITTER_LEDGER_ORDERER=<IP_ADDRESS> CORE_PEER_ADDRESS=<IP_ADDRESS> peer channel join -b myc1.block

If you are seeing errors while using the node SDK, make sure you have the correct versions of node.js and npm installed on your machine. You want node v6.9.5 and npm v3.10.10.

If you ran through the automated channel create/join process (i.e. did not comment out channel_test.sh in the docker-compose-gettingstarted.yml), then channel - myc1 - and genesis block - myc1.block - have already been created and exist on your machine. As a result, if you proceed to execute the manual steps in your cli container:

CORE_PEER_COMMITTER_LEDGER_ORDERER=orderer:7050 peer channel create -c myc1

Then you will run into an error similar to:

<EXACT_TIMESTAMP>       UTC [msp] Sign -> DEBU 064 Sign: digest: 5ABA6805B3CDBAF16C6D0DCD6DC439F92793D55C82DB130206E35791BCF18E5F
Error: Got unexpected status: BAD_REQUEST
Usage:
  peer channel create [flags]

This occurs because you are attempting to create a channel named myc1, and this channel already exists! There are two options. Try issuing the peer channel create command with a different channel name - myc2. For example:

CORE_PEER_COMMITTER_LEDGER_ORDERER=orderer:7050 peer channel create -c myc2

Then join:

CORE_PEER_COMMITTER_LEDGER_ORDERER=orderer:7050 CORE_PEER_ADDRESS=peer0:7051 peer channel join -b myc2.block

If you do choose to create a new channel, and want to run deploy/invoke/query with the node.js programs, you also need to edit the “channelID” parameter in the config.json file to match the new channel’s name. For example:

{
   "chainName":"fabric-client1",
   "chaincodeID":"mycc",
   "channelID":"myc2",
   "goPath":"../../test/fixtures",
   "chaincodePath":"github.com/example_cc",

OR, if you want your channel called - myc1 -, remove your docker containers and then follow the same commands in the Manually create and join peers to a new channel section.

Clean up¶

Shut down your containers:

docker-compose -f docker-compose-gettingstarted.yml down

Helpful Docker tips¶

Remove a specific docker container:

docker rm <containerID>

Force removal:

docker rm -f <containerID>

Remove all docker containers:

docker rm -f $(docker ps -aq)

This will merely kill docker containers (i.e. stop the process). You will not lose any images.

Remove an image:

docker rmi <imageID>

Forcibly remove:

docker rmi -f <imageID>

Remove all images:

docker rmi -f $(docker images -q)

Chaincode interfaces¶

A chaincode implements the Chaincode Interface that supports two methods: * Init * Invoke

Dependencies¶

The import statement lists a few dependencies for the chaincode to compile successfully. * fmt – contains Println for debugging/logging. * errors – standard go error format. * shim – contains the definitions for the chaincode interface and the chaincode stub, which you are required to interact with the ledger.

Chaincode APIs¶

When the Init or Invoke function of a chaincode is called, the fabric passes the stub shim.ChaincodeStubInterface parameter and the chaincode returns a pb.Response. This stub can be used to call APIs to access to the ledger services, transaction context, or to invoke other chaincodes.

The current APIs are defined in the shim package, and can be generated with the following command:

godoc github.com/hyperledger/fabric/core/chaincode/shim

However, it also includes functions from chaincode.pb.go (protobuffer functions) that are not intended as public APIs. The best practice is to look at the function definitions in chaincode.go and and the examples directory.

Response¶

The chaincode response comes in the form of a protobuffer.

message Response {

    // A status code that should follow the HTTP status codes.
    int32 status = 1;

    // A message associated with the response code.
    string message = 2;

    // A payload that can be used to include metadata with this response.
    bytes payload = 3;

}

The chaincode will also return events. Message events and chaincode events.

messageEvent {

    oneof Event {

    //Register consumer sent event
    Register register = 1;

    //producer events common.
    Block block = 2;
    ChaincodeEvent chaincodeEvent = 3;
    Rejection rejection = 4;

    //Unregister consumer sent events
    Unregister unregister = 5;

    }

}

messageChaincodeEvent {

    string chaincodeID = 1;
    string txID = 2;
    string eventName = 3;
    bytes payload = 4;

}

Once developed and deployed, there are two ways to interact with the chaincode - through an SDK or the CLI. The steps for CLI are described below. For SDK interaction, refer to the balance transfer samples. Note: This SDK interaction is covered in the Getting Started section.

Command Line Interfaces¶

To view the currently available CLI commands, execute the following:

# this assumes that you have correctly set the GOPATH variable and cloned the Fabric codebase into that path
cd /opt/gopath/src/github.com/hyperledger/fabric
build /bin/peer

You will see output similar to the example below. (NOTE: rootcommand below is hardcoded in main.go. Currently, the build will create a peer executable file).

Usage:
      peer [flags]
      peer [command]

    Available Commands:
      version     Print fabric peer version.
      node        node specific commands.
      channel     channel specific commands.
      chaincode   chaincode specific commands.
      logging     logging specific commands


    Flags:
      --logging-level string: Default logging level and overrides, see core.yaml for full syntax
      --test.coverprofile string: Done (default “coverage.cov)
      -v, --version: Display current version of fabric peer server
    Use "peer [command] --help" for more information about a command.

The peer command supports several subcommands and flags, as shown above. To facilitate its use in scripted applications, the peer command always produces a non-zero return code in the event of command failure. Upon success, many of the subcommands produce a result on stdout as shown in the table below:

--peer-defaultchain=true

Deploy a chaincode¶

[WIP] - the CLI commands need to be refactored based on the new deployment model. Channel Create and Channel Join will remain the same.

Overview¶

Logging in the peer application and in the shim interface to chaincodes is programmed using facilities provided by the github.com/op/go-logging package. This package supports

• Logging control based on the severity of the message
• Logging control based on the software module generating the message
• Different pretty-printing options based on the severity of the message

All logs are currently directed to stderr, and the pretty-printing is currently fixed. However global and module-level control of logging by severity is provided for both users and developers. There are currently no formalized rules for the types of information provided at each severity level, however when submitting bug reports the developers may want to see full logs down to the DEBUG level.

In pretty-printed logs the logging level is indicated both by color and by a 4-character code, e.g, “ERRO” for ERROR, “DEBU” for DEBUG, etc. In the logging context a module is an arbitrary name (string) given by developers to groups of related messages. In the pretty-printed example below, the logging modules “peer”, “rest” and “main” are generating logs.

16:47:09.634 [peer] GetLocalAddress -> INFO 033 Auto detected peer address: 9.3.158.178:7051
16:47:09.635 [rest] StartOpenchainRESTServer -> INFO 035 Initializing the REST service...
16:47:09.635 [main] serve -> INFO 036 Starting peer with id=name:"vp1" , network id=dev, address=9.3.158.178:7051, discovery.rootnode=, validator=true

An arbitrary number of logging modules can be created at runtime, therefore there is no “master list” of modules, and logging control constructs can not check whether logging modules actually do or will exist. Also note that the logging module system does not understand hierarchy or wildcarding: You may see module names like “foo/bar” in the code, but the logging system only sees a flat string. It doesn’t understand that “foo/bar” is related to “foo” in any way, or that “foo/*” might indicate all “submodules” of foo.

peer¶

The logging level of the peer command can be controlled from the command line for each invocation using the --logging-level flag, for example

peer node start --logging-level=debug

The default logging level for each individual peer subcommand can also be set in the core.yaml file. For example the key logging.node sets the default level for the node subcommmand. Comments in the file also explain how the logging level can be overridden in various ways by using environment varaibles.

Logging severity levels are specified using case-insensitive strings chosen from

CRITICAL | ERROR | WARNING | NOTICE | INFO | DEBUG

The full logging level specification for the peer is of the form

[<module>[,<module>...]=]<level>[:[<module>[,<module>...]=]<level>...]

A logging level by itself is taken as the overall default. Otherwise, overrides for individual or groups of modules can be specified using the

<module>[,<module>...]=<level>

syntax. Examples of specifications (valid for all of --logging-level, environment variable and core.yaml settings):

info                                       - Set default to INFO
warning:main,db=debug:chaincode=info       - Default WARNING; Override for main,db,chaincode
chaincode=info:main=debug:db=debug:warning - Same as above

Go chaincodes¶

The standard mechanism to log within a chaincode application is to integrate with the logging transport exposed to each chaincode instance via the peer. The chaincode shim package provides APIs that allow a chaincode to create and manage logging objects whose logs will be formatted and interleaved consistently with the shim logs.

As independently executed programs, user-provided chaincodes may technically also produce output on stdout/stderr. While naturally useful for “devmode”, these channels are normally disabled on a production network to mitigate abuse from broken or malicious code. However, it is possible to enable this output even for peer-managed containers (e.g. “netmode”) on a per-peer basis via the CORE_VM_DOCKER_ATTACHSTDOUT=true configuration option.

Once enabled, each chaincode will receive its own logging channel keyed by its container-id. Any output written to either stdout or stderr will be integrated with the peer’s log on a per-line basis. It is not recommended to enable this for production.

LogDebug, LogInfo, LogNotice, LogWarning, LogError, LogCritical

DEBUG, INFO, NOTICE, WARNING, ERROR, CRITICAL

(c *ChaincodeLogger) Debug(args ...interface{})
(c *ChaincodeLogger) Info(args ...interface{})
(c *ChaincodeLogger) Notice(args ...interface{})
(c *ChaincodeLogger) Warning(args ...interface{})
(c *ChaincodeLogger) Error(args ...interface{})
(c *ChaincodeLogger) Critical(args ...interface{})

(c *ChaincodeLogger) Debugf(format string, args ...interface{})
(c *ChaincodeLogger) Infof(format string, args ...interface{})
(c *ChaincodeLogger) Noticef(format string, args ...interface{})
(c *ChaincodeLogger) Warningf(format string, args ...interface{})
(c *ChaincodeLogger) Errorf(format string, args ...interface{})
(c *ChaincodeLogger) Criticalf(format string, args ...interface{})

var logger = shim.NewLogger("myChaincode")

func main() {

    logger.SetLevel(shim.LogInfo)

    logLevel, _ := shim.LogLevel(os.Getenv("SHIM_LOGGING_LEVEL"))
    shim.SetLoggingLevel(logLevel)
    ...
}

Table of contents¶

Part I: Elements of the architecture relevant to Hyperledger Fabric v1

1. System architecture

2. Basic workflow of transaction endorsement

3. Endorsement policies

   Part II: Post-v1 elements of the architecture

4. Ledger checkpointing (pruning)

1. System architecture¶

The blockchain is a distributed system consisting of many nodes that communicate with each other. The blockchain runs programs called chaincode, holds state and ledger data, and executes transactions. The chaincode is the central element as transactions are operations invoked on the chaincode. Transactions have to be “endorsed” and only endorsed transactions may be committed and have an effect on the state. There may exist one or more special chaincodes for management functions and parameters, collectively called system chaincodes.

2. Basic workflow of transaction endorsement¶

In the following we outline the high-level request flow for a transaction.

Remark: Notice that the following protocol *does not assume that all transactions are deterministic, i.e., it allows for non-deterministic transactions.*

2.4. The ordering service delivers a transactions to the peers¶

When an event deliver(seqno, prevhash, blob) occurs and a peer has applied all state updates for blobs with sequence number lower than seqno, a peer does the following:

• It checks that the blob.endorsement is valid according to the policy of the chaincode (blob.tran-proposal.chaincodeID) to which it refers.
• In a typical case, it also verifies that the dependencies (blob.endorsement.tran-proposal.readset) have not been violated meanwhile. In more complex use cases, tran-proposal fields in endorsement may differ and in this case endorsement policy (Section 3) specifies how the state evolves.

Verification of dependencies can be implemented in different ways, according to a consistency property or “isolation guarantee” that is chosen for the state updates. Serializability is a default isolation guarantee, unless chaincode endorsement policy specifies a different one. Serializability can be provided by requiring the version associated with every key in the readset to be equal to that key’s version in the state, and rejecting transactions that do not satisfy this requirement.

• If all these checks pass, the transaction is deemed valid or committed. In this case, the peer marks the transaction with 1 in the bitmask of the PeerLedger, applies blob.endorsement.tran-proposal.writeset to blockchain state (if tran-proposals are the same, otherwise endorsement policy logic defines the function that takes blob.endorsement).
• If the endorsement policy verification of blob.endorsement fails, the transaction is invalid and the peer marks the transaction with 0 in the bitmask of the PeerLedger. It is important to note that invalid transactions do not change the state.

Note that this is sufficient to have all (correct) peers have the same state after processing a deliver event (block) with a given sequence number. Namely, by the guarantees of the ordering service, all correct peers will receive an identical sequence of deliver(seqno, prevhash, blob) events. As the evaluation of the endorsement policy and evaluation of version dependencies in readset are deterministic, all correct peers will also come to the same conclusion whether a transaction contained in a blob is valid. Hence, all peers commit and apply the same sequence of transactions and update their state in the same way.

Illustration of the transaction flow (common-case path).

Figure 1. Illustration of one possible transaction flow (common-case path).

3. Endorsement policies¶

4 (post-v1). Validated ledger and PeerLedger checkpointing (pruning)¶

Endorsement policy design¶

Endorsement policies have two main components: - a principal - a threshold gate

A principal P identifies the entity whose signature is expected.

A threshold gate T takes two inputs: an integer t (the threshold) and a list of n principals or gates; this gate essentially captures the expectation that out of those n principals or gates, t are requested to be satisfied.

For example: - T(2, 'A', 'B', 'C') requests a signature from any 2 principals out of ‘A’, ‘B’ or ‘C’; - T(1, 'A', T(2, 'B', 'C')) requests either one signature from principal A or 1 signature from B and C each.

Endorsement policy syntax in the CLI¶

In the CLI, a simple language is used to express policies in terms of boolean expressions over principals.

A principal is described in terms of the MSP that is tasked to validate the identity of the signer and of the role that the signer has within that MSP. Currently, two roles are supported: member and admin. Principals are described as MSP.ROLE, where MSP is the MSP ID that is required, and ROLE is either one of the two strings member and admin. Examples of valid principals are 'Org0.admin' (any administrator of the Org0 MSP) or 'Org1.member' (any member of the Org1 MSP).

The syntax of the language is:

EXPR(E[, E...])

where EXPR is either AND or OR, representing the two boolean expressions and E is either a principal (with the syntax described above) or another nested call to EXPR.

For example: - AND('Org1.member', 'Org2.member', 'Org3.member') requests 1 signature from each of the three principals - OR('Org1.member', 'Org2.member') requests 1 signature from either one of the two principals - OR('Org1.member', AND('Org2.member', 'Org3.member')) requests either one signature from a member of the Org1 MSP or 1 signature from a member of the Org2 MSP and 1 signature from a member of the Org3 MSP.

Specifying endorsement policies for a chaincode¶

Using this language, a chaincode deployer can request that the endorsements for a chaincode be validated against the specified policy. NOTE - the default policy requires one signature from a member of the DEFAULT MSP). This is used if a policy is not specified in the CLI.

The policy can be specified at deploy time using the -P switch, followed by the policy.

For example:

peer chaincode deploy -C testchainid -n mycc -p github.com/hyperledger/fabric/examples/chaincode/go/chaincode_example02 -c '{"Args":["init","a","100","b","200"]}' -P "AND('Org1.member', 'Org2.member')"

This command deploys chaincode mycc on chain testchainid with the policy AND('Org1.member', 'Org2.member').

Future enhancements¶

In this section we list future enhancements for endorsement policies: - alongside the existing way of identifying principals by their relationship with an MSP, we plan to identify principals in terms of the Organization Unit (OU) expected in their certificates; this is useful to express policies where we request signatures from any identity displaying a valid certificate with an OU matching the one requested in the definition of the principal. - instead of the syntax AND(., .) we plan to move to a more intuitive syntax . AND . - we plan to expose generalized threshold gates in the language as well alongside AND (which is the special n-out-of-n gate) and OR (which is the special 1-out-of-n gate)

Prerequisites¶

• Go 1.7+ installation or later
• GOPATH environment variable is set correctly

Install¶

To install the fabric-ca command:

# go get github.com/hyperledger/fabric-ca

The Fabric CA CLI¶

The following shows the fabric-ca CLI usage:

# fabric-ca
fabric-ca client       - client related commands
fabric-ca server       - server related commands
fabric-ca cfssl        - all cfssl commands

For help, type "fabric-ca client", "fabric-ca server", or "fabric-ca cfssl".

The fabric-ca server and fabric-ca client commands are discussed below.

If you would like to enable debug-level logging (for server or client), set the FABRIC_CA_DEBUG environment variable to true.

Since fabric-ca is built on top of CFSSL, the fabric-ca cfssl commands are available but are not discussed in this document. See CFSSL for more information.

Postgres SSL Configuration¶

Basic instructions for configuring SSL on Postgres server: 1. In postgresql.conf, uncomment SSL and set to “on” (SSL=on) 2. Place Certificate and Key files Postgress data directory.

Instructions for generating self-signed certificates for: https://www.postgresql.org/docs/9.1/static/ssl-tcp.html

Note: Self-signed certificates are for testing purposes and should not be used in a production environment

Postgres Server - Require Client Certificates 1. Place certificates of the certificate authorities (CAs) you trust in the file root.crt in the Postgres data directory 2. In postgresql.conf, set “ssl_ca_file” to point to the root cert of client (CA cert) 3. Set the clientcert parameter to 1 on the appropriate hostssl line(s) in pg_hba.conf.

For more details on configuring SSL on the Postgres server, please refer to the following Postgres documentation: https://www.postgresql.org/docs/9.4/static/libpq-ssl.html

MySQL SSL Configuration¶

Basic instructions for configuring SSL on MySQL server:

1. Open or create my.cnf file for the server. Add or un-comment the lines below in [mysqld] section. These should point to the key and certificates for the server, and the root CA cert.

Instruction on creating server and client side certs: http://dev.mysql.com/doc/refman/5.7/en/creating-ssl-files-using-openssl.html

[mysqld] ssl-ca=ca-cert.pem ssl-cert=server-cert.pem ssl-key=server-key.pem

Can run the following query to confirm SSL has been enabled.

mysql> SHOW GLOBAL VARIABLES LIKE ‘have_%ssl’;

Should see:

+---------------+-------+   | Variable_name | Value |   +---------------+-------+   | have_openssl  | YES   |   | have_ssl      | YES   |   +---------------+-------+

2. After the server-side SSL configuration is finished, the next step is to create a user who has a privilege to access the MySQL server over SSL. For that, log in to the MySQL server, and type:

mysql> GRANT ALL PRIVILEGES ON . TO ‘ssluser’@’%’ IDENTIFIED BY ‘password’ REQUIRE SSL; mysql> FLUSH PRIVILEGES;

If you want to give a specific ip address from which the user will access the server change the ‘%’ to the specific ip address.

MySQL Server - Require Client Certificates Options for secure connections are similar to those used on the server side.

• ssl-ca identifies the Certificate Authority (CA) certificate. This option, if used, must specify the same certificate used by the server.
• ssl-cert identifies the client public key certificate.
• ssl-key identifies the client private key.

Suppose that you want to connect using an account that has no special encryption requirements or was created using a GRANT statement that includes the REQUIRE SSL option. As a recommended set of secure-connection options, start the MySQL server with at least –ssl-cert and –ssl-key, and invoke the fabric-ca server with ca_certfiles option set in the fabric-ca server file.

To require that a client certificate also be specified, create the account using the REQUIRE X509 option. Then the client must also specify the proper client key and certificate files or the MySQL server will reject the connection. CA cert, client cert, and client key are all required for the fabric-ca server.

How is the confidentiality of transactions and business logic achieved?¶

The security module works in conjunction with the membership service module to provide access control service to any data recorded and business logic deployed on a chain network.

When a code is deployed on a chain network, whether it is used to define a business contract or an asset, its creator can put access control on it so that only transactions issued by authorized entities will be processed and validated by chain validators.

Raw transaction records are permanently stored in the ledger. While the contents of non-confidential transactions are open to all participants, the contents of confidential transactions are encrypted with secret keys known only to their originators, validators, and authorized auditors. Only holders of the secret keys can interpret transaction contents.

  ##### What if none of the stakeholders of a business contract are validators? In some business scenarios, full confidentiality of contract logic may be required – such that only contract counterparties and auditors can access and interpret their chaincode. Under these scenarios, counter parties would need to spin off a new child chain with only themselves as validators.

Getting a Linux Foundation account¶

In order to participate in the development of the Hyperledger Fabric project, you will need a Linux Foundation account. You will need to use your LF ID to access to all the Hyperledger community development tools, including Gerrit, Jira and the Wiki (for editing, only).

Getting help¶

If you are looking for something to work on, or need some expert assistance in debugging a problem or working out a fix to an issue, our community is always eager to help. We hang out on Chat, IRC (#hyperledger on freenode.net) and the mailing lists. Most of us don’t bite :grin: and will be glad to help. The only silly question is the one you don’t ask. Questions are in fact a great way to help improve the project as they highlight where our documentation could be clearer.

Requirements and Use Cases¶

We have a Requirements WG that is documenting use cases and from those use cases deriving requirements. If you are interested in contributing to this effort, please feel free to join the discussion in chat.

Reporting bugs¶

If you are a user and you find a bug, please submit an issue using JIRA. Please try to provide sufficient information for someone else to reproduce the issue. One of the project’s maintainers should respond to your issue within 24 hours. If not, please bump the issue with a comment and request that it be reviewed. You can also post to the #fabric-maintainers channel in chat.

Fixing issues and working stories¶

Review the issues list and find something that interests you. You could also check the “help-wanted” list. It is wise to start with something relatively straight forward and achievable, and that no one is already assigned. If no one is assigned, then assign the issue to yourself. Please be considerate and rescind the assignment if you cannot finish in a reasonable time, or add a comment saying that you are still actively working the issue if you need a little more time.

Working with a local clone and Gerrit¶

We are using Gerrit to manage code contributions. If you are unfamiliar, please review this document before proceeding.

After you have familiarized yourself with Gerrit, and maybe played around with the lf-sandbox project, you should be ready to set up your local development environment.

Next, try building the project in your local development environment to ensure that everything is set up correctly.

Logging control describes how to tweak the logging levels of various components within the Fabric. Finally, every source file needs to include a license header: modified to include a copyright statement for the principle author(s).

What makes a good change request?¶

• One change at a time. Not five, not three, not ten. One and only one. Why? Because it limits the blast area of the change. If we have a regression, it is much easier to identify the culprit commit than if we have some composite change that impacts more of the code.
• Include a link to the JIRA story for the change. Why? Because a) we want to track our velocity to better judge what we think we can deliver and when and b) because we can justify the change more effectively. In many cases, there should be some discussion around a proposed change and we want to link back to that from the change itself.
• Include unit and integration tests (or changes to existing tests) with every change. This does not mean just happy path testing, either. It also means negative testing of any defensive code that it correctly catches input errors. When you write code, you are responsible to test it and provide the tests that demonstrate that your change does what it claims. Why? Because without this we have no clue whether our current code base actually works.
• Unit tests should have NO external dependencies. You should be able to run unit tests in place with go test or equivalent for the language. Any test that requires some external dependency (e.g. needs to be scripted to run another component) needs appropriate mocking. Anything else is not unit testing, it is integration testing by definition. Why? Because many open source developers do Test Driven Development. They place a watch on the directory that invokes the tests automagically as the code is changed. This is far more efficient than having to run a whole build between code changes.
• Minimize the lines of code per CR. Why? Maintainers have day jobs, too. If you send a 1,000 or 2,000 LOC change, how long do you think it takes to review all of that code? Keep your changes to < 200-300 LOC if possible. If you have a larger change, decompose it into multiple independent changess. If you are adding a bunch of new functions to fulfill the requirements of a new capability, add them separately with their tests, and then write the code that uses them to deliver the capability. Of course, there are always exceptions. If you add a small change and then add 300 LOC of tests, you will be forgiven;-) If you need to make a change that has broad impact or a bunch of generated code (protobufs, etc.). Again, there can be exceptions.
• Write a meaningful commit message. Include a meaningful 50 (or less) character title, followed by a blank line, followed my a more comprehensive description of the change. Be sure to include the JIRA identifier corresponding to the change (e.g. [FAB-1234]). This can be in the title but should also be in the body of the commit message.

e.g.

[FAB-1234] fix foobar() panic

Fix [FAB-1234] added a check to ensure that when foobar(foo string) is called,
that there is a non-empty string argument.

Finally, be responsive. Don’t let a change request fester with review comments such that it gets to a point that it requires a rebase. It only further delays getting it merged and adds more work for you - to remediate the merge conflicts.

Coding guidelines¶

Be sure to check out the language-specific style guides before making any changes. This will ensure a smoother review.

Communication¶

We use RocketChat for communication and Google Hangouts™ for screen sharing between developers. Our development planning and prioritization is done in JIRA, and we take longer running discussions/decisions to the mailing list.

Maintainers¶

The project’s maintainers are responsible for reviewing and merging all patches submitted for review and they guide the over-all technical direction of the project within the guidelines established by the Hyperledger Project’s Technical Steering Committee (TSC).

Legal stuff¶

Note: Each source file must include a license header for the Apache Software License 2.0. A template of that header can be found here.

We have tried to make it as easy as possible to make contributions. This applies to how we handle the legal aspects of contribution. We use the same approach—the Developer’s Certificate of Origin 1.1 (DCO)—that the Linux® Kernel community uses to manage code contributions.

We simply ask that when submitting a patch for review, the developer must include a sign-off statement in the commit message.

Here is an example Signed-off-by line, which indicates that the submitter accepts the DCO:

Signed-off-by: John Doe <john.doe@hisdomain.com>

You can include this automatically when you commit a change to your local git repository using git commit -s.

Creating a Linux Foundation ID¶

1. Go to the Linux Foundation ID website.
2. Select the option I need to create a Linux Foundation ID.
3. Fill out the form that appears:
4. Open your email account and look for a message with the subject line: “Validate your Linux Foundation ID email”.
5. Open the received URL to validate your email address.
6. Verify the browser displays the message You have successfully    validated your e-mail address.
7. Access Gerrit by selecting Sign In:
8. Use your Linux Foundation ID to Sign In:

Configuring Gerrit to Use SSH¶

Gerrit uses SSH to interact with your Git client. A SSH private key needs to be generated on the development machine with a matching public key on the Gerrit server.

If you already have a SSH key-pair, skip this section.

As an example, we provide the steps to generate the SSH key-pair on a Linux environment. Follow the equivalent steps on your OS.

1. Create a key-pair, enter:

ssh-keygen -t rsa -C "John Doe john.doe@example.com"

Note: This will ask you for a password to protect the private key as it generates a unique key. Please keep this password private, and DO NOT enter a blank password.

The generated key-pair is found in: ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub.

1. Add the private key in the id_rsa file in your key ring, e.g.:

ssh-add ~/.ssh/id_rsa

Once the key-pair has been generated, the public key must be added to Gerrit.

Follow these steps to add your public key id_rsa.pub to the Gerrit account:

1. Go to Gerrit.
2. Click on your account name in the upper right corner.
3. From the pop-up menu, select Settings.
4. On the left side menu, click on SSH Public Keys.
5. Paste the contents of your public key ~/.ssh/id_rsa.pub and click Add key.

Note: The id_rsa.pub file can be opened with any text editor. Ensure that all the contents of the file are selected, copied and pasted into the Add SSH key window in Gerrit.

Note: The ssh key generation instructions operate on the assumtion that you are using the default naming. It is possible to generate multiple ssh Keys and to name the resulting files differently. See the ssh-keygen documentation for details on how to do that. Once you have generated non-default keys, you need to configure ssh to use the correct key for Gerrit. In that case, you need to create a ~/.ssh/config file modeled after the one below.

host gerrit.hyperledger.org
 HostName gerrit.hyperledger.org
 IdentityFile ~/.ssh/id_rsa_hyperledger_gerrit
 User <LFID>

where is your Linux Foundation ID and the value of IdentityFile is the name of the public key file you generated.

Warning: Potential Security Risk! Do not copy your private key ~/.ssh/id_rsa Use only the public ~/.ssh/id_rsa.pub.

Checking Out the Source Code¶

1. Ensure that SSH has been set up properly. See Configuring Gerrit to Use SSH for details.
2. Clone the repository with your Linux Foundation ID ():

git clone ssh://<LFID>@gerrit.hyperledger.org:29418/fabric fabric

You have successfully checked out a copy of the source code to your local machine.

Overview¶

Through the v0.6 release, the development environment utilized Vagrant running an Ubuntu image, which in turn launched Docker containers as a means of ensuring a consistent experience for developers who might be working with varying platforms, such as MacOSX, Windows, Linux, or whatever. Advances in Docker have enabled native support on the most popular development platforms: MacOSX and Windows. Hence, we have reworked our build to take full advantage of these advances. While we still maintain a Vagrant based approach that can be used for older versions of MacOSX and Windows that Docker does not support, we strongly encourage that the non-Vagrant development setup be used.

Note that while the Vagrant-based development setup could not be used in a cloud context, the Docker-based build does support cloud platforms such as AWS, Azure, Google and IBM to name a few. Please follow the instructions for Ubuntu builds, below.

Prerequisites¶

• Git client
• Go - 1.7 or later (for releases before v1.0, 1.6 or later)
• For MacOSX, Xcode must be installed
• Docker - 1.12 or later
• Pip
• (MacOSX) you may need to install gnutar, as MacOSX comes with bsdtar as the default, but the build uses some gnutar flags. You can use Homebrew to install it as follows:

brew install gnu-tar --with-default-names

• (only if using Vagrant) - Vagrant - 1.7.4 or later
• (only if using Vagrant) - VirtualBox - 5.0 or later
• BIOS Enabled Virtualization - Varies based on hardware
• Note: The BIOS Enabled Virtualization may be within the CPU or Security settings of the BIOS

pip, behave and docker-compose¶

pip install --upgrade pip
pip install behave nose docker-compose
pip install -I flask==0.10.1 python-dateutil==2.2 pytz==2014.3 pyyaml==3.10 couchdb==1.0 flask-cors==2.0.1 requests==2.4.3 pyOpenSSL==16.2.0 sha3==0.2.1

Steps¶

git config --get core.autocrlf

git config --global core.autocrlf false

cd $GOPATH/src
mkdir -p github.com/hyperledger
cd github.com/hyperledger

git clone ssh://LFID@gerrit.hyperledger.org:29418/fabric && scp -p -P 29418 LFID@gerrit.hyperledger.org:hooks/commit-msg fabric/.git/hooks/

cd $GOPATH/src/github.com/hyperledger/fabric/devenv
vagrant up

vagrant ssh

Building the fabric¶

Once you have all the dependencies installed, and have cloned the repository, you can proceed to build and test the fabric.

Notes¶

NOTE: Any time you change any of the files in your local fabric directory (under $GOPATH/src/github.com/hyperledger/fabric), the update will be instantly available within the VM fabric directory.

NOTE: If you intend to run the development environment behind an HTTP Proxy, you need to configure the guest so that the provisioning process may complete. You can achieve this via the vagrant-proxyconf plugin. Install with vagrant plugin install vagrant-proxyconf and then set the VAGRANT_HTTP_PROXY and VAGRANT_HTTPS_PROXY environment variables before you execute vagrant up. More details are available here: https://github.com/tmatilai/vagrant-proxyconf/

NOTE: The first time you run this command it may take quite a while to complete (it could take 30 minutes or more depending on your environment) and at times it may look like it’s not doing anything. As long you don’t get any error messages just leave it alone, it’s all good, it’s just cranking.

NOTE to Windows 10 Users: There is a known problem with vagrant on Windows 10 (see mitchellh/vagrant#6754). If the vagrant up command fails it may be because you do not have the Microsoft Visual C++ Redistributable package installed. You can download the missing package at the following address: http://www.microsoft.com/en-us/download/details.aspx?id=8328

Running the unit tests¶

Use the following sequence to run all unit tests

cd $GOPATH/src/github.com/hyperledger/fabric
make unit-test

To run a specific test use the -run RE flag where RE is a regular expression that matches the test case name. To run tests with verbose output use the -v flag. For example, to run the TestGetFoo test case, change to the directory containing the foo_test.go and call/excecute

go test -v -run=TestGetFoo

Running Node.js Unit Tests¶

You must also run the Node.js unit tests to insure that the Node.js client SDK is not broken by your changes. To run the Node.js unit tests, follow the instructions here.

Running Behave BDD Tests¶

Note: currently, the behave tests must be run from within in the Vagrant environment. See the devenv setup instructions if you have not already set up your Vagrant environment.

Behave tests will setup networks of peers with different security and consensus configurations and verify that transactions run properly. To run these tests

cd $GOPATH/src/github.com/hyperledger/fabric
make behave

Some of the Behave tests run inside Docker containers. If a test fails and you want to have the logs from the Docker containers, run the tests with this option:

cd $GOPATH/src/github.com/hyperledger/fabric/bddtests
behave -D logs=Y

Building on Z¶

To make building on Z easier and faster, this script is provided (which is similar to the setup file provided for vagrant). This script has been tested only on RHEL 7.2 and has some assumptions one might want to re-visit (firewall settings, development as root user, etc.). It is however sufficient for development in a personally-assigned VM instance.

To get started, from a freshly installed OS:

sudo su
yum install git
mkdir -p $HOME/git/src/github.com/hyperledger
cd $HOME/git/src/github.com/hyperledger
git clone http://gerrit.hyperledger.org/r/fabric
source fabric/devenv/setupRHELonZ.sh

From this point, you can proceed as described above for the Vagrant development environment.

cd $GOPATH/src/github.com/hyperledger/fabric
make peer unit-test behave

Building on Power Platform¶

Development and build on Power (ppc64le) systems is done outside of vagrant as outlined here. For ease of setting up the dev environment on Ubuntu, invoke this script as root. This script has been validated on Ubuntu 16.04 and assumes certain things (like, development system has OS repositories in place, firewall setting etc) and in general can be improvised further.

To get started on Power server installed with Ubuntu, first ensure you have properly setup your Host’s GOPATH environment variable. Then, execute the following commands to build the fabric code:

mkdir -p $GOPATH/src/github.com/hyperledger
cd $GOPATH/src/github.com/hyperledger
git clone http://gerrit.hyperledger.org/r/fabric
sudo ./fabric/devenv/setupUbuntuOnPPC64le.sh
cd $GOPATH/src/github.com/hyperledger/fabric
make dist-clean all

Git-review¶

There’s a very useful tool for working with Gerrit called git-review. This command-line tool can automate most of the ensuing sections for you. Of course, reading the information below is also highly recommended so that you understand what’s going on behind the scenes.

Sandbox project¶

We have created a sandbox project to allow developers to familiarize themselves with Gerrit and our workflows. Please do feel free to use this project to experiment with the commands and tools, below.

Getting deeper into Gerrit¶

A comprehensive walk-through of Gerrit is beyond the scope of this document. There are plenty of resources available on the Internet. A good summary can be found here. We have also provided a set of Best Practices that you may find helpful.

Working with a local clone of the repository¶

To work on something, whether a new feature or a bugfix:

1. Open the Gerrit Projects page

2. Select the project you wish to work on.

3. Open a terminal window and clone the project locally using the Clone with git hook URL. Be sure that ssh is also selected, as this will make authentication much simpler:

   git clone ssh://LFID@gerrit.hyperledger.org:29418/fabric && scp -p -P 29418 LFID@gerrit.hyperledger.org:hooks/commit-msg fabric/.git/hooks/
   

Note: if you are cloning the fabric project repository, you will want to clone it to the $GOPATH/src/github.com/hyperledger directory so that it will build, and so that you can use it with the Vagrant development environment.

4. Create a descriptively-named branch off of your cloned repository

cd fabric
git checkout -b issue-nnnn

5. Commit your code. For an in-depth discussion of creating an effective commit, please read this document.

git commit -s -a

Then input precise and readable commit msg and submit.

6. Any code changes that affect documentation should be accompanied by corresponding changes (or additions) to the documentation and tests. This will ensure that if the merged PR is reversed, all traces of the change will be reversed as well.

Submitting a Change¶

Currently, Gerrit is the only method to submit a change for review. Please review the `guidelines <changes.md>`__ for making and submitting a change.

[remote "gerrit"]
    url = ssh://<USERNAME>@gerrit.hyperledger.org:29418/fabric.git
    fetch = +refs/heads/*:refs/remotes/gerrit/*

$ cd <your code dir>
$ git review

cd <your clone dir>
git push origin HEAD:refs/for/master

Counting objects: 3, done.
Writing objects: 100% (3/3), 306 bytes | 0 bytes/s, done.
Total 3 (delta 0), reused 0 (delta 0)
remote: Processing changes: new: 1, refs: 1, done
remote:
remote: New Changes:
remote:   https://gerrit.hyperledger.org/r/6 Test commit
remote:
To ssh://LFID@gerrit.hyperledger.org:29418/fabric
* [new branch]      HEAD -> refs/for/master

Adding reviewers¶

Optionally, you can add reviewers to your change.

To specify a list of reviewers via the command line, add %r=reviewer@project.org to your push command. For example:

git push origin HEAD:refs/for/master%r=rev1@email.com,r=rev2@notemail.com

Alternatively, you can auto-configure GIT to add a set of reviewers if your commits will have the same reviewers all at the time.

To add a list of default reviewers, open the :file:.git/config file in the project directory and add the following line in the [ branch “master” ] section:

[branch "master"] #.... push =
HEAD:refs/for/master%r=rev1@email.com,r=rev2@notemail.com`

Make sure to use actual email addresses instead of the @email.com and @notemail.com addressses. Don’t forget to replace origin with your git remote name.

Reviewing Using Gerrit¶

• Add: This button allows the change submitter to manually add names of people who should review a change; start typing a name and the system will auto-complete based on the list of people registered and with access to the system. They will be notified by email that you are requesting their input.
• Abandon: This button is available to the submitter only; it allows a committer to abandon a change and remove it from the merge queue.
• Change-ID: This ID is generated by Gerrit (or system). It becomes useful when the review process determines that your commit(s) have to be amended. You may submit a new version; and if the same Change-ID header (and value) are present, Gerrit will remember it and present it as another version of the same change.
• Status: Currently, the example change is in review status, as indicated by “Needs Verified” in the upper-left corner. The list of Reviewers will all emit their opinion, voting +1 if they agree to the merge, -1 if they disagree. Gerrit users with a Maintainer role can agree to the merge or refuse it by voting +2 or -2 respectively.

Notifications are sent to the email address in your commit message’s Signed-off-by line. Visit your Gerrit dashboard, to check the progress of your requests.

The history tab in Gerrit will show you the in-line comments and the author of the review.

Viewing Pending Changes¶

Find all pending changes by clicking on the All --> Changes link in the upper-left corner, or open this link.

If you collaborate in multiple projects, you may wish to limit searching to the specific branch through the search bar in the upper-right side.

Add the filter project:fabric to limit the visible changes to only those from the Hyperledger Fabric Project.

List all current changes you submitted, or list just those changes in need of your input by clicking on My --> Changes or open this link

Change Requirements¶

This section contains guidelines for submitting code changes for review. For more information on how to submit a change using Gerrit, please see Gerrit.

Changes are submitted as Git commits. Each commit must contain:

• a short and descriptive subject line that is 72 characters or fewer, followed by a blank line.
• a change description with your logic or reasoning for the changes, followed by a blank line
• a Signed-off-by line, followed by a colon (Signed-off-by:)
• a Change-Id identifier line, followed by a colon (Change-Id:). Gerrit won’t accept patches without this identifier.

A commit with the above details is considered well-formed.

All changes and topics sent to Gerrit must be well-formed. Informationally, commit messages must include:

• what the change does,
• why you chose that approach, and
• how you know it works – for example, which tests you ran.

Commits must build cleanly when applied in top of each other, thus avoiding breaking bisectability. Each commit must address a single identifiable issue and must be logically self-contained.

For example: One commit fixes whitespace issues, another renames a function and a third one changes the code’s functionality. An example commit file is illustrated below in detail:

A short description of your change with no period at the end

You can add more details here in several paragraphs, but please keep each line
width less than 80 characters. A bug fix should include the issue number.

Fix Issue # 7050.

Change-Id: IF7b6ac513b2eca5f2bab9728ebd8b7e504d3cebe1
Signed-off-by: Your Name <commit-sender@email.address>

Each commit must contain the following line at the bottom of the commit message:

Signed-off-by: Your Name <your@email.address>

The name in the Signed-off-by line and your email must match the change authorship information. Make sure your :file:.git/config is set up correctly. Always submit the full set of changes via Gerrit.

When a change is included in the set to enable other changes, but it will not be part of the final set, please let the reviewers know this.

Browsing the Git Tree¶

Visit Gerrit then select Projects --> List --> SELECT-PROJECT --> Branches. Select the branch that interests you, click on gitweb located on the right-hand side. Now, gitweb loads your selection on the Git web interface and redirects appropriately.

Watching a Project¶

Visit Gerrit, then select Settings, located on the top right corner. Select Watched Projects and then add any projects that interest you.

Commit Messages¶

Gerrit follows the Git commit message format. Ensure the headers are at the bottom and don’t contain blank lines between one another. The following example shows the format and content expected in a commit message:

Brief (no more than 50 chars) one line description.

Elaborate summary of the changes made referencing why (motivation), what was changed and how it was tested. Note also any changes to documentation made to remain consistent with the code changes, wrapping text at 72 chars/line.

The Gerrit server provides a precommit hook to autogenerate the Change-Id which is one time use.

Recommended reading: How to Write a Git Commit Message

Avoid Pushing Untested Work to a Gerrit Server¶

To avoid pushing untested work to Gerrit.

Check your work at least three times before pushing your change to Gerrit. Be mindful of what information you are publishing.

Keeping Track of Changes¶

• Set Gerrit to send you emails:
• Gerrit will add you to the email distribution list for a change if a developer adds you as a reviewer, or if you comment on a specific Patch Set.
• Opening a change in Gerrit’s review interface is a quick way to follow that change.
• Watch projects in the Gerrit projects section at Gerrit, select at least New Changes, New Patch Sets, All Comments and Submitted Changes.

Always track the projects you are working on; also see the feedback/comments mailing list to learn and help others ramp up.

Topic branches¶

Topic branches are temporary branches that you push to commit a set of logically-grouped dependent commits:

To push changes from REMOTE/master tree to Gerrit for being reviewed as a topic in TopicName use the following command as an example:

$ git push REMOTE HEAD:refs/for/master/TopicName

The topic will show up in the review :abbr:UI and in the Open Changes List. Topic branches will disappear from the master tree when its content is merged.

Creating a Cover Letter for a Topic¶

You may decide whether or not you’d like the cover letter to appear in the history.

1. To make a cover letter that appears in the history, use this command:

git commit --allow-empty

Edit the commit message, this message then becomes the cover letter. The command used doesn’t change any files in the source tree.

2. To make a cover letter that doesn’t appear in the history follow these steps:

• Put the empty commit at the end of your commits list so it can be ignored

  without having to rebase.

• Now add your commits

git commit ...
git commit ...
git commit ...

• Finally, push the commits to a topic branch. The following command is an example:

git push REMOTE HEAD:refs/for/master/TopicName

If you already have commits but you want to set a cover letter, create an empty commit for the cover letter and move the commit so it becomes the last commit on the list. Use the following command as an example:

git rebase -i HEAD~#Commits

Be careful to uncomment the commit before moving it. #Commits is the sum of the commits plus your new cover letter.

Finding Available Topics¶

$ ssh -p 29418 gerrit.hyperledger.org gerrit query \ status:open project:fabric branch:master \
| grep topic: | sort -u

• `gerrit.hyperledger.org <>`__ Is the current URL where the project is hosted.
• status Indicates the topic’s current status: open , merged, abandoned, draft, merge conflict.
• project Refers to the current name of the project, in this case fabric.
• branch The topic is searched at this branch.
• topic The name of an specific topic, leave it blank to include them all.
• sort Sorts the found topics, in this case by update (-u).

Downloading or Checking Out a Change¶

In the review UI, on the top right corner, the Download link provides a list of commands and hyperlinks to checkout or download diffs or files.

We recommend the use of the git review plugin. The steps to install git review are beyond the scope of this document. Refer to the git review documentation for the installation process.

To check out a specific change using Git, the following command usually works:

git review -d CHANGEID

If you don’t have Git-review installed, the following commands will do the same thing:

git fetch REMOTE refs/changes/NN/CHANGEIDNN/VERSION \ && git checkout FETCH_HEAD

For example, for the 4th version of change 2464, NN is the first two digits (24):

git fetch REMOTE refs/changes/24/2464/4 \ && git checkout FETCH_HEAD

Using Draft Branches¶

You can use draft branches to add specific reviewers before you publishing your change. The Draft Branches are pushed to refs/drafts/master/TopicName

The next command ensures a local branch is created:

git checkout -b BRANCHNAME

The next command pushes your change to the drafts branch under TopicName:

git push REMOTE HEAD:refs/drafts/master/TopicName

Using Sandbox Branches¶

You can create your own branches to develop features. The branches are pushed to the refs/sandbox/USERNAME/BRANCHNAME location.

These commands ensure the branch is created in Gerrit’s server.

git checkout -b sandbox/USERNAME/BRANCHNAME
git push --set-upstream REMOTE HEAD:refs/heads/sandbox/USERNAME/BRANCHNAME

Usually, the process to create content is:

• develop the code,
• break the information into small commits,
• submit changes,
• apply feedback,
• rebase.

The next command pushes forcibly without review:

git push REMOTE sandbox/USERNAME/BRANCHNAME

You can also push forcibly with review:

git push REMOTE HEAD:ref/for/sandbox/USERNAME/BRANCHNAME

Updating the Version of a Change¶

During the review process, you might be asked to update your change. It is possible to submit multiple versions of the same change. Each version of the change is called a patch set.

Always maintain the Change-Id that was assigned. For example, there is a list of commits, c0...c7, which were submitted as a topic branch:

git log REMOTE/master..master

c0
...
c7

git push REMOTE HEAD:refs/for/master/SOMETOPIC

After you get reviewers’ feedback, there are changes in c3 and c4 that must be fixed. If the fix requires rebasing, rebasing changes the commit Ids, see the rebasing section for more information. However, you must keep the same Change-Id and push the changes again:

git push REMOTE HEAD:refs/for/master/SOMETOPIC

This new push creates a patches revision, your local history is then cleared. However you can still access the history of your changes in Gerrit on the review UI section, for each change.

It is also permitted to add more commits when pushing new versions.

Rebasing¶

Rebasing is usually the last step before pushing changes to Gerrit; this allows you to make the necessary Change-Ids. The Change-Ids must be kept the same.

• squash: mixes two or more commits into a single one.
• reword: changes the commit message.
• edit: changes the commit content.
• reorder: allows you to interchange the order of the commits.
• rebase: stacks the commits on top of the master.

Rebasing During a Pull¶

Before pushing a rebase to your master, ensure that the history has a consecutive order.

For example, your REMOTE/master has the list of commits from a0 to a4; Then, your changes c0...c7 are on top of a4; thus:

git log --oneline REMOTE/master..master

a0
a1
a2
a3
a4
c0
c1
...
c7

If REMOTE/master receives commits a5, a6 and a7. Pull with a rebase as follows:

git pull --rebase REMOTE master

This pulls a5-a7 and re-apply c0-c7 on top of them:

$ git log --oneline REMOTE/master..master
a0
...
a7
c0
c1
...
c7

Getting Better Logs from Git¶

Use these commands to change the configuration of Git in order to produce better logs:

git config log.abbrevCommit true

The command above sets the log to abbreviate the commits’ hash.

git config log.abbrev 5

The command above sets the abbreviation length to the last 5 characters of the hash.

git config format.pretty oneline

The command above avoids the insertion of an unnecessary line before the Author line.

To make these configuration changes specifically for the current Git user, you must add the path option --global to config as follows:

Coding Golang¶

We code in Go™ and strictly follow the best practices and will not accept any deviations. You must run the following tools against your Go code and fix all errors and warnings: - golint - go vet - goimports