Write some Software

Technical brief Build a simple web-based management interface allowing a user to trigger restarts of various services and reboots of remote servers Background We have built and maintain a number of websites and services which are hosted across a range of cloud­based environments. Some of the older systems, when facing sudden traffic spikes or conflict with other services, result in services and even servers locking up and becoming unresponsive to the point of SSH log­ins failing. While all these choke points will be eradicated in future, we are in need of a simple interim solution that will allow a non­technical resource to restart services and servers are required. Requirements System mechanisms Monitoring of service and server states Your code is not responsible for this role. Restarting of system services Your application will need to be able to execute a service restart of any specified daemon on the remote server. This is to be done via SSH using a key file for authentication. All servers are Linux based and are using the upstart “service” command. Rebooting a virtual server Both Amazon EC2 instances and DigitalOcean droplets can be power­cycled via the hosting provider’s APIs. Client libraries exist for use in PHP and are easy to implement. Rebooting an AWS Elasticache node Some of our products rely on Amazon’s Elasticache server ­ a Memcached implementation. These cache nodes can be rebooted using Amazon’s AWS API. Configuration storage Configuration “recipes” are to be stored in individual human readable files. JSON, YAML or XML are all acceptable format choices. Each recipe is a discrete set of steps that your system must follow if we determine that a particular server is in an error state. Configuration entry types 1. Service restart or shell command Any command that can be executed on the targeted server via SSH 2. Server restart Using the relevant hosting API, perform a power­cycle on the targeted server instance 3. Pause Used to allow the system user to confirm if preceding actions have resolved the problem A typical recipe would play out as follows: Failed Webserver X 1. Restart MySQL 2. Restart Apache 3. Pause for user to check on success 4. If not successful, restart the server User interface The system must have a simple web interface. The user is presented with a list of all available servers loaded from the configuration files. The user selects the server that needs to be fixed and clicks a button to activate the repair sequence. Feedback is given as each step begins to execute and again at the completion of each step. Where necessary the process pauses to await user confirmation of success or failure. No user authentication is required for the interface as access to the server will be restricted to LAN users. Security At such time as you need them for development and testing, you will be provided with SSH keys, Amazon EC2 keys and DigitalOcean keys for testing servers. Your configuration files must provide for dynamic assignment of names and storage paths of these key files.