Firefox Marketplace Documentation¶

Setting up the Backend¶

If you want to work on the backend, the API, or the payments infrastructure, visit our Marketplace Backend documentation. It will explain how to set up Docker as it automates the setup of the Marketplace environment.

Setting up the Frontend¶

If you want to work on the consumer-facing Marketplace frontend, visit our Marketplace Frontend documentation. It only takes three commands to get things up and running. You don’t even need to setup a backend, although you could if you wanted your own personal playground.

List of Services and Repositories¶

Marketplace Servers¶

Marketplace’s servers are outlined below For specifics of our production configuration, please see the Services documentation (only visible to Mozilla employees).

• dev: updated each commit.
• alt dev: updated each commit. Used for testing disruptive development features.
• stage: updated when tags are made. This is as similar to production as possible. Used for testing features before they go to production. Uses real money for payments.
• payments alt: updated each commit. Used for testing dispruptive payments features. Uses real money for payments.
• production: updated as per the push schedule. The production servers are the only ones with any uptime expectations.

Requirements¶

• Mac OS X or Linux.
• A github account (set in step 3).
• packages: python2.7, python-pip, curl, python-dev

1. Install Docker¶

Install Docker.

brew install caskroom/cask/brew-cask
brew cask install virtualbox
brew install docker
brew install docker-machine
brew install docker-compose

curl -sSL https://get.docker.com/ | sh

sudo usermod -aG docker ubuntu

sudo pip install docker-compose

2. Build dependencies¶

docker-machine create --driver virtualbox --virtualbox-memory 2048 mkt

eval "$(docker-machine env mkt)"

git clone https://github.com/mozilla/marketplace-env.git
cd marketplace-env
python link-sources.py --root $(dirname $(pwd))

export COMPOSE_FILE=`pwd`/docker-compose.yml
export COMPOSE_PROJECT_NAME=mkt  # String prepended to every container.

sudo sh -c "echo $(docker-machine ip mkt 2>/dev/null)  mp.dev >> /etc/hosts"

sudo sh -c "echo 127.0.0.1  mp.dev >> /etc/hosts"

3. Build and run¶

Now that the marketplace github repos are cloned and linked we can start creating the virtual machine and docker containers.

Let’s pull down the docker images and build the containers:

docker-compose pull

Next, start the containers:

docker-compose up -d

When everything is running open up a browser to http://mp.dev

4. Shutting down and restarting¶

On the Marketplace team we have found it good practice to shut down docker at the end of each work day.

docker-compose stop
docker-machine stop mkt

docker-machine start mkt
docker-compose up -d

docker-compose stop

docker-compose start -d

Issues¶

Come talk to us on irc://irc.mozilla.org/marketplace if you have questions, issues, or compliments.

docker-compose Commands¶

This is not a full list of commands, for that see the docker-compose docs <https://docs.docker.com/compose/>, just notable ones.

docker-compose up [-d]

Use this if you would like all the service logs in the foreground run:

docker-compose up  # Ctrl-c here will shutdown all services.

docker-compose logs

View log output from a project’s container.

docker Commands¶

For docs see: https://docs.docker.com/

How do I run commands?¶

It’s easiest to think of the containers as Linux machines you can shell into to run various commands.

Docker labels each container with a prefix of ‘mkt’ (as defined by COMPOSE_PROJECT_NAME environment variable) and a numeric suffix, which will be 1 unless you are running multiple of the same container.

So you can shell into any container by running (for zamboni, e.g.):

docker exec -ti mkt_zamboni_1 /bin/bash

From there you can run migrations:

schematic migrations/

or the unit tests:

./manage.py test --noinput -s --logging-clear-handlers

Another option is to spawn up a new instance of the container temporarily to run a command, e.g.:

docker-compose run --rm zamboni schematic migrations/

How do I update python/node package deps (rebuild the container)?¶

1. First stop the machine with your OS-specific instructions.
2. Then pull the newest images from Docker Hub:

docker-compose pull [project]

3. Then start the machine with your OS-specific instructions.

How do I add an admin in Zamboni with docker?¶

Simply run this command replacing name@email.com with the email of the user you’ve recently logged-in as:

docker-compose run --rm zamboni python manage.py addusertogroup name@email.com 1

How do I upgrade docker?¶

Upgrade a machine to the latest version of Docker:

docker-machine upgrade

Other Environment Variables¶

Please be aware that other parts of the site infrastructure can be affected by environment variables. Some examples:

• If you want to use custom Django settings, you can set DJANGO_SETTINGS_MODULE

Serving With Nginx¶

Marketplace is designed to be an app accessible at one domain, hitting Nginx.

Behind the scenes Nginx will proxy to the other servers on your behalf.

Most developers are using Nginx to serve out the multiple services. Your configuration may look something like this:

You can find an example configuration file in our Docker repository.

Default Ports¶

By default, the services listen to the following ports:

External services¶

The Marketplace interacts with multiple remote services that are not under the control of the Marketplace team.

Notes:

• Zippy: is a reference implemention of the Marketplace Payments Specification to enable easy testing and development.
• Bango and Boku: do not provide test instances.
• Boku: uses a different set of integrator keys for different servers, please see the internal docs on mana.
• Firefox Accounts: native flow on a device connects to the production Firefox Accounts. The web based flow connects to the servers as noted above.

Setup¶

Setting up an instance of the Marketplace frontend for development is very simple. Note, you will need Node and NPM set up on your machine:

git clone git@github.com:mozilla/fireplace
npm install
make install
make serve

This will clone the project, install dependencies, copy assets from Bower into the project, create a require.js configuration with paths set up, and start a local webserver on port 8675 by default.

Firefox OS¶

To develop on Firefox OS, we recommend using Firefox’s WebIDE. Follow the instructions on connecting a device.

To generate a package (i.e., a Marketplace app to install on the device):

make package

Then install the package onto the device using WebIDE.

Android¶

To develop on Android, you’ll need to make sure that the Android device can access your local development instance.

If you’re running your development environment on a vm then being able to accessing it from an android device can be a challenge. To solve this problem on android there’s a couple of options.

• Use a proxy with DNS spoofing like Charles proxy http://www.charlesproxy.com/. Using a recent android phone you can configure a proxy under the advanced settings of a wi-fi connection. This doesn’t require root.
• Root your phone and push a hosts file to it with adb.

Setup¶

To develop payments locally we would recommend that you install Webpay, Solitude, Zippy and Spartacus. This will give you a complete end to end flow of how payments would work but you will be missing:

• actual carrier or credit card charges
• actual carrier interaction, such as user identification

If you want that then you’ll need to set up integration with a real payment provider, such as Bango. That is discussed in the solitude docs.

The following assumes that you have a successfully working Spartacus, Webpay, Solitude, Zippy and Zamboni installed. This document discusses how you need to configure them all to interact.

If you are unsure on some of these settings:

• Most other developers have these settings done and are happy to help
• Each project’s settings files are in GitHub with examples and documentation

from base import *

MARKETPLACE_OATH = {'key': 'a', 'secret': 'b'}

Configuring devices¶

Out of the box, Firefox OS only ships with settings that let you make payments against the production server. If you want to pay with a hosted dev or stage server then you’ll need to put some custom settings on your B2G device. See the developer docs if you want to host your own WebPay.

git clone git://github.com/mozilla/webpay.git

cd webpay
cp ezboot.ini-dist ezboot.ini

ezboot flash

ezboot setup

ezboot mkt_certs --dev --certs_path ~/Downloads/certdb.tmp/

ezboot install_mkt --dev

ezboot install --manifest https://marketplace.allizom.org/manifest.webapp

git clone git://github.com/mozilla-b2g/gaia.git
cd gaia
git checkout --track -b origin/v1.4 origin/v1.4

git checkout origin/v1.4
git pull

pref("dom.payment.skipHTTPSCheck", true);
pref("dom.identity.enabled", true);
pref("toolkit.identity.debug", true);

pref("dom.payment.provider.1.name", "firefoxmarketdev");
pref("dom.payment.provider.1.description", "marketplace-dev.allizom.org");
pref("dom.payment.provider.1.uri", "https://marketplace-dev.allizom.org/mozpay/?req=");
pref("dom.payment.provider.1.type", "mozilla-dev/payments/pay/v1");
pref("dom.payment.provider.1.requestMethod", "GET");

pref("dom.payment.provider.2.name", "firefoxmarketstage");
pref("dom.payment.provider.2.description", "marketplace.allizom.org");
pref("dom.payment.provider.2.uri", "https://marketplace.allizom.org/mozpay/?req=");
pref("dom.payment.provider.2.type", "mozilla-stage/payments/pay/v1");
pref("dom.payment.provider.2.requestMethod", "GET");

pref("dom.payment.provider.3.name", "firefoxmarketlocal");
pref("dom.payment.provider.3.description", "localhost");
pref("dom.payment.provider.3.uri", "http://localhost:2601/mozpay/?req=");
pref("dom.payment.provider.3.type", "mozilla-local/payments/pay/v1");
pref("dom.payment.provider.3.requestMethod", "GET");

pref("dom.payment.provider.4.name", "firefoxmarketalt");
pref("dom.payment.provider.4.description", "payments-alt.allizom.org");
pref("dom.payment.provider.4.uri", "https://payments-alt.allizom.org/mozpay/?req=");
pref("dom.payment.provider.4.type", "mozilla-alt/payments/pay/v1");
pref("dom.payment.provider.4.requestMethod", "GET");

make clean profile

./flash.sh

cd gaia
adb shell "stop b2g"
adb push profile/user.js /data/local/
adb reboot

Create a New Package¶

These directions assume you have a copy of the Marketplace frontend checked out.

If you want to make a production package (i.e., using the production API), simply run from the root of your Fireplace checkout:

make package

If you are building a packaged app containing an iframed Marketplace (which is currently running in production as of February 2015) as opposed to a true packaged app containing the actual Marketplace and all its assets, you’ll instead want to run:

make iframe_package

Packages will be output to /package/builds/. You can also make packages for other servers. These are pre-configured within config.js:

SERVER=altdev make package
SERVER=dev make package
SERVER=paymentsalt make package
SERVER=stage make package

If you are running a local backend server (e.g., http://mp.dev), you can configure and build a custom package if you wanted. Open config.js and add an entry to packageConfig. It might look like:

'local': {
    domain: 'http://mp.dev',
    media_url: 'http://mp.dev',
    name: 'Local',
    origin: 'app://packaged.local.firefox.com'
}

To test your package, use Firefox’s WebIDE.

Put the New Package on the Marketplace¶

These directions assume you have the site permissions (meaning: you own the app in the Marketplace) to do this and you want to put it on production. Adjust the URL below if you want to put it on dev or stage.

1. Edit the app on the Marketplace
2. Upload the newly made package
3. Make sure that Multiple Network Information and Network Information are checked
4. Save your changes. The file will be in the approval queue.

Approve the New Package on the Marketplace¶

These directions assume you have the reviewer permissions to do this.

1. Load the app in the Reviewer Tools
2. Click Push to Public
3. Enter a short message in comments and click OK

Preload a New Package in Firefox OS¶

For steps 4 to 8, we have a helper script to download the package and generate Etags.

These directions assume you have forked gaia.

1. File a bug for Gaia saying a new Marketplace is incoming. They won’t land a PR without a bug filed.

2. Check out a new branch

3. Navigate to the Marketplace files:

   cd apps/marketplace.firefox.com
   

4. Run in a separate window and make note of the output of these commands. Note that if you are building yogafire and not fireplace you’ll want to use marketplace-package.webapp from the same URL below.

   curl 'https://marketplace.firefox.com/packaged.webapp'
   
   curl -I 'https://marketplace.firefox.com/packaged.webapp' | grep ETag
   

5. The first command above will be a blob of JSON including a package_path. Copy that package path (eg. https://marketplace.firefox.com/downloads/file/251994/marketplace-20140331214114.zip). The second command will have an ETag header (eg. a10ca98addc3785e92ead363281c425bd7114b84a4162f50096593b76a7ac2c3) - copy that for later.

6. Replace the current marketplace with the new package using the package_path you copied above. Example:

   curl 'https://marketplace.firefox.com/downloads/file/251994/marketplace-20140331214114.zip' > application.zip
   

7. Get the package’s new ETag:

   curl -I 'https://marketplace.firefox.com/downloads/file/251994/marketplace-20140331214114.zip' | grep ETag
   

8. Modify metadata.json and replace the etag and packageEtag fields with the two ETags from above. Note that the extra escaped quotes are not a typo. Your diff will look something like:

   + "etag": "\"a10ca98addc3785e92ead363281c425bd7114b84a4162f50096593b76a7ac2c3\"",
   + "packageEtag": "\"bf5f4736daffaf7982c872efc4beb38f440d5d84a6fb3f82c5d434cca6abd4d5\"",
   

9. Commit your changes and make a pull request. Put the bug number from step 1 in the title of the bug. For example: Bug 100000 - New Marketplace; 20140501

10. Return to the bug you filed in step 1. Click Add Attachment -> Paste text as attachment and paste in your pull request URL (eg. https://github.com/mozilla-b2g/gaia/pull/18845). Under flags request review? from :julienw or :fabrice

11. Done!

Testing Your Package on Device¶

Do you have that tingle in your gut that says you should test our the package on an actual phone before making a PR? Or maybe you made some changes to the certs and you want to make sure they work? Well, you’re in luck, because it’s super easy (this will erase everything on your phone):

1. Make sure you’re in the root of your Gaia repository
2. Make sure you can see your Flame device with adb devices
3. Run make reset-gaia to build your gaia and push to the device

Flashing the Latest Build¶

Have ADB installed. You can learn how to install adb on MDN.

Connect your phone to your machine with a USB cable. If you run adb devices you should see a device ID.

git clone https://github.com/Mozilla-TWQA/B2G-flash-tool
cd B2G-flash-tool
./flash_pvt.py --help
# Shallow flash an engineering build off the master branch onto the
# Flame v188 device.  In theory you won't lose anything...
./flash_pvt.py -d flame-kk -v mozilla-central --eng -g -G --keep_profile

Enabling Dev and Stage Certificates¶

Installing Dev and Stage on Engineering Builds¶

Engineering builds (as of 16 Oct 2014) have old Dev and Stage Marketplace apps installed, and they cannot be removed.

To work around this, we need root access on the engineering image in order to push custom prefs (e.g., for payments), and the UI and apps from the user build.

To do so, flash a full engineering image (referred to as “images” as opposed to “gaia+gecko”). Then flash a user build of just “gaia+gecko” of the same build. Since this the user build, the developer menu is hidden by default. To remedy this:

• Go to “Device Information -> More information”
• Scroll to bottom and enable developer menu
• Enable USB Debugging in the dev menu
• Check console enabled
• Enable Marketplace reviewer certs
• Reboot

Accessing Your Local Marketplace From Device¶

If you want to access your local Marketplace Backend on a device, you’ll need to proxy the internal virtual server through a public IP and bind that IP to a host on device.

If you run Apache on port 80 which is the default on many systems, you can add this to your config to proxy. Adjust the internal IP address of the virtual server as necessary.:

Listen 80

<VirtualHost *:80>
    ServerName mp.dev
    ProxyPreserveHost On
    ProxyRequests off
    ProxyPass / http://192.168.59.103/
    ProxyPassReverse / http://192.168.59.103/
</VirtualHost>

If you run nginx on port 80 then you can use a config like this. Again, you may need to adjust the proxied IP:

http {
    server {
        listen       80 default;
        server_name  mp.dev;

        location / {
            # Pass public connections to the internal
            # Docker / VirtualBox server.
            proxy_pass http://192.168.59.103/;
            proxy_set_header Host $host;
        }
    }
}

When running Docker and serving on your public / network IP (sucha s 10.0.0.1), ensure USB debugging is enabled on your device, plug it in, and use the bind command:

bin/mkt bind

This will edit the /system/etc/hosts file on the device so that you can access http://mp.dev.

If you have multiple network devices, the command will prompt you for the one to bind to. Run bin/mkt bind --help for details.

Prefs File for Payments Testing¶

Here’s an example prefs file for payments testing: https://gist.github.com/muffinresearch/9a7c3d3d632a9a9922f0

Push this to your device with:

adb push path/to/custom-prefs.js /data/local/user.js

Then reboot for the changes to take effect:

adb reboot

Installing Packaged Marketplaces¶

Apps such as Dev or Stage or PaymentsAlt, are unlisted on the production Marketplace. Though, Metaplace can be installed which allows you to install the apps from the “jump” menu.

Or you can go directly to the app page from the browser on the device:

• Dev: https://marketplace.firefox.com/app/mkt-dev
• Stage: https://marketplace.firefox.com/app/mkt-stage
• PaymentsAlt: https://marketplace.firefox.com/app/marketplace-payments-alt

Device Not Being Recognized¶

If your Tarako device is not recognized, add the vendor ID to the ADB USB configuration:

echo "0x1782" > ~/.android/adb_usb.ini
adb kill-server
adb start-server