HAT - Hatimeria Automation Toolkit

Installs locally php based projects with docker and deploy to remote servers node and php applications.

Requirements

• NodeJS (https://nodejs.org/en/download/current/ - v12.x) - pick any suitable installer from this page.

Optionally for managing local workspace with docker download Docker.

Resources

• Development
• Deployment
• Changelog
• Docker
• Blackfire
• Composer
• Wildcard domains
• FakeSendmail & mailtrap.io
• Known issues
• Custom project-specific commands
• Using HAT events
• Performance
• Magento 2 installation tips
• Using YAML configuration file
• Deploy repositories - working with production/stages servers
• Slack notifications
• Node Versions
• PHP-FPM

Environment setup

HAT may read some environment variables and use them as default values for options. Set them in your .bashrc or .zshrc file.

• VAULT_ADDR - default address to vault in format https://vault.domain.com:8200
• VAULT_TOKEN - token to vault
• NODE_ENV - used for setting log level, available options are debug or silly
• DEPLOY_REPO_RE - regex used when detecting deploy repository context
• DEFAULT_ADMIN - set default admin user name, if not present hatadmin will be used
• DEFAULT_ADMIN_PASS - set admin user password, if not present password123123 will be used
• DEFAULT_ADMIN_EMAIL - set admin user email, if not present [email protected] will be used

Deploying projects with HAT

Deploy repository

All deployments are done in deployment repository context. Deploy repo are folders with a specific structure that defines servers and applications that can be deployed. By default HAT detectes if given path is a deploy repo according to the following regex: '\/(deploy-[a-z\-]*|config)\/(prod|stage|feature)\/(.*)$'. You can change this by setting DEPLOY_REPO_RE environment variable. However, after the match only 2 elements will be used as projectEnvType and projectEnv so make sure match of your REGEXP provides at least 4 results.

The structure of typical deploy repo looks like this:

• app

  • default.yml
  • shop.yml

• prod

  • alpha
    • app
      • default.yml
      • shop.yml

• stage

  • alpha
    • app
      • default.yml
      • shop.yml
  • beta
    • app
      • default.yml
      • shop.yml

• app - folder contains files with default values of flags. It is read and merged for every command

  • default.yml - optional file with default values for all types and envrironments
  • shop.yml - optional file with flag values for shop application; any flags defined here will overwrite the ones find in defult.yml

• prod - this folder contains a list of server considered as production. This name is required

  • alpha - contains a set of application for the same environment; the name is unrestricted, it also does not mean all application live in the same physical server, this is just a group name that should contain all applications required for your project to run fully
    • default.yml - default flags for all applications in prod/alpha context. Overwrites any flags read from files in app folder
    • shop.yml - default flags for shop application; overwrites flags from app folder and default.yml

• stage or feature - this folder contains a list of server considered as production. This name is required

  • alpha, beta - the same rules apply as in the prod/alpha

Deploy procedure

Deployment is run after executing hat deploy (or hat i for short) command in one of the specific environment folders (ie. prod/alpha, stage/beta). If no --app param is provided to tell which application to work with the command tries to guest by searching files in the current folders, default.yml file is ommited. If only 1 file is present it will be used. If more than 1 file is present then and an error will be thrown as HAT cannot determine which application should be deployed.

The second required param is --type which tells what project type the given application is. It can be provided in yml files or passed from cli.

For any additional help call hat i --help or hat i --app shop --help.

More information about deploy repo and deployment process are in:

• Deployment
• Deploy repository
• Events

Loggin deployment

It is possible to send deployment information to external services, ie. New Relic. There are 3 flags that need to be provided for the message to be sent:

• --record-deployment - boolean value stating to send information
• --record-deplyoment-host - host where the information need to be sent
• --record-deplyoment-path - path in the service

The message is a JSON object:

{
  "appName":  "${projectName}_${app}",
  "user":  "system username who calls the command",
  "revision":  "provided gitTreeish flag",
  "description":  "Summary of deployment (app type, user, project domain)",
}

To utilize it with New Relic you need to build a middleware that will change appName to New Relic application ID. This can be done via AWS lambda function or your own service.

Local project installation

Quick start (clean)

To install a clean project via HAT:

$ cd path/to/your/project/acme
$ hat project li [...options]

Command should output acme.test (acme - depends on your target folder name) as domain and other web credentials when project can be reached in the web browser.

Supported project types (for both - clean installation and installation from DB dump):

• Magento 1
• Magento 2
• WordPress (3.7 or later)
• Symfony

Quick start (from DB dump)

Clone your project repository.

Copy production database dump to your project root dir under prod.sql name.

From your project folder you want to run in Docker - run the following command:

$ cd path/to/your/project/acme
$ hat project li --db-dump prod.sql

Project-specific and DB configuration for your containers is being stored in ~/.hat_config/config_CONTAINER_NAME.json. It will be loaded as your default values so you don't need to use them again.

Available Docker images

• hatimeria/php70:latest (PHP 7.0 and MySQL 5.7) - use it like --image-version php70:latest, db user is admin, no password
• hatimeria/php70:nodb (PHP 7.0) - use it like --image-version php70:nodb, db user is admin, no password
• hatimeria/php71:latest (PHP 7.1 and MySQL 5.7) - use it like --image-version php71:latest, db user is admin, no password
• hatimeria/php71:nodb (PHP 7.1) - use it like --image-version php70:nodb
• hatimeria/php72:latest (PHP 7.2 and MySQL 5.7) - use it like --image-version php72:latest, db user is admin, no password
• hatimeria/php72:nodb (PHP 7.2) - use it like --image-version php72:nodb
• hatimeria/php73:latest (PHP 7.3 and MySQL 5.7) - use it like --image-version php73:latest, db user is admin, no password
• hatimeria/php73:nodb (PHP 7.3) - use it like --image-version php73:nodb

To use one of these images - you need to set with --image-version flag like --image-version php71:nodb

Setting up database

There are 3 ways you can set up the database for the project:

• use image with mysql server installed in it and connect through localhost
• use separate container with mysql and connect to it. This option is available when use --db-use-container flag. For more info go to DATABASE
• start a project with --docker-compose flag to create separate container just for database for this project. For more info go to DATABASE

Useful commands

• Login to container (under www-data user)

$ hat container ssh

• Login to container under root user

$ hat container ssh-root

• Resume container after reboot

$ hat container resume

• Stop container

$ hat container stop

Application Debug

Magento image has pre-configured XDebug settings:

zend_extension=xdebug.so
xdebug.remote_enable=On
xdebug.remote_port=10000
xdebug.remote_handler=dbgp
xdebug.remote_connect_back=Off
xdebug.remote_host=10.254.254.254

It's also required to install a Chrome extension to manage XDebug modes (debugging, profiling and tracing)

You need to set your IDE settings to work with 10000 xdebug port!

If you still can't get XDebug working with Docker Beta - you should try this fix.

As stated you need to run on your host:

sudo ifconfig lo0 alias 10.254.254.254

Due to performance all hatimeria images comes with commented zend_extension line so xdebug is not loaded. To start xdebug use command

hat container xdebug-enable

To stop it use

hat container xdebug-disable

Updating HAT

Each developer has to update Magento Docker image along with HAT! It can be done by calling docker image pull command or using --force-image-update flag on hat project li command

Extra help

To get extra help on using HAT application or its commands - run one of the following commands:

$ hat -h
$ hat [command] -h

To check available options and check their default values for running project with HAT, run the following command from your project folder:

$ cd path/to/your/project/acme
$ hat project i -h

Using HTTPS

To use HTTPS within your projects - you can use these flags:

• --allow-https-domain - this flag shows to "domain-fix" operation to use HTTPS for domain (in Magento case - use HTTPS for secure domain)
• --force-https-domain - (if true - sets --allow-https-domain=true automatically) mostly applicable for Magento. Sets secure and unsecure to HTTPS for "domain-fix" operation
• --enable-https-domain - enables Let's encrypt certificates generation for the project (works with Docker containers only)

Debugging HAT

To see more logs switch node env to debug.

export NODE_ENV=debug

or

export NODE_ENV=silly

You can also use --verbose-exec flag to see what commands are executed

Populating the magento instance

Using install script

Each project type offers installation by the native install script:

• Magento 1.X - uses instal.php script in the project root folder
• Magento 2 - uses bin/magento setup:install command
• Wordpress - uses wp-clip.php core install command
• Symfony - uses doctrine:schema:create command (with dropping current schema)
• Node - does not offer an installator

From DB dump

Optionally db dump can be used to populate the instance

$ hat project li --db-dump relative/path/to/dump.sql ...options...

Running Magerun commands (for Magento-v1 only)

Developers are able to run Magerun commands directly from HAT on the host machine.

In case if Magerun library is not installed - HAT will download it to the project root folder automatically.

To run Magerun command - you need to run it like the following:

$ hat magento run <magerun-command>

If you won't specify <magerun-command> part - you will get a list of available Magerun commands.

The change default HAT settings - you have to specify them BEFORE the Magerun command, for example, specifying container name should be done like hat magento-run --container magento_name <magerun-command>).

All Magerun-related options MUST be specified after Magerun command, for example, hat magento-run <hat-options...> <magerun-command> --<magerun-option> ... (everything after <magerun-command> string will be passed to the Magerun library).

Storing (almost) sensitive data with HAT

If you want to store some credentials, like GitHub token, repo.magento.com auth data etc - you can do it with HAT.

To store simple config data (data will be stored in ~/.hat/config.json file), use it like this:

$ hat container set-config key value

To store data in a separate file (for example data) - you need to pass an extra --scope [scope] flag:

$ hat container set-config --scope data key value

Storing passwords and tokens with HAT

To store any username/password or token with HAT, you need to provide those values like the following example (auth data will be stored as a string):

$ hat container set-auth-config service-url.com oauth-token

If you want to store a username/password pair, just pass another value to HAT command (in this case auth data will be stored as an object):

$ hat container set-auth-config service-url.com username password

All the above examples - will be stored for "global" usage. In case, if you have project-specific data - you can pass --project-domain flag:

$ hat container set-auth-config --project-domain test.com service-url.com username password

HAT will try to load project-specific data first, if it fails - HAT will try to use auth data from global scope.