From c7536c911643fa16bb78d69dd1d63f94f1a2c92f Mon Sep 17 00:00:00 2001 From: Donggang Cao Date: Fri, 8 Apr 2016 15:48:37 +0800 Subject: [PATCH] update README --- README.md | 123 ++++++++++++++++++------------------------------------ 1 file changed, 40 insertions(+), 83 deletions(-) diff --git a/README.md b/README.md index 628afbe..8008fa8 100644 --- a/README.md +++ b/README.md @@ -2,38 +2,28 @@ http://docklet.unias.org -## intro +## Intro -Docklet is an operating system for mini-datacener. Its goal is to help -multi-user share cluster resources effectively. Unlike the "application -framework oriented" cluster manager such as Mesos and Yarn, Docklet is -**user oriented**. In Docklet, every user has their own private -**virtual cluster (vcluster)**, which consists of a number of virtual -Linux container nodes distributed over the physical cluster. Every -vcluster is separated from others and can be operated like a real -physical cluster. Therefore, most applications, especially those -requiring a cluster environment, can run in vcluster seamlessly. +Docklet is a cloud operating system for mini-datacener. Its goal is to +help multi-user share cluster resources effectively. In Docklet, every +user has their own private **virtual cluster (vcluster)**, which +consists of a number of virtual Linux container nodes distributed over +the physical cluster. Each vcluster is separated from others and can be +operated like a real physical cluster. Therefore, most applications, +especially those requiring a cluster environment, can run in vcluster +seamlessly. -Docklet provides a base image for creating virtual nodes. This image has -pre-installed a lot of mainstream development tools and frameworks, -including gcc/g++, openjdk, python3, R, MPI, scala, ruby, php, node.js, -texlive, mpich2, spark, -scipy/numpy/matplotlib/pandas/sympy/scikit-learn, jupyter notebook, etc. -Users can get a ready vcluster with just one click within 1 second. +Users manage and use their vcluster all through web. The only client +tool needed is a modern web browser supporting HTML5, like Safari, +Firefox, or Chrome. The integrated *jupyter notebook* provides a web +**Workspace**. In the Workspace, users can code, debug, test, +and runn their programs, even visualize the outputs online. +Therefore, it is ideal for data analysis and processing. -The users are free to install their specific software in their vcluster. -Docklet supports operating through **web terminal**. Users can do their -work as an administrator working on a console. The base image system is -ubuntu. The recommended way of installing new software is by -**apt-get**. - -The users manage and use their vcluster all through web. The only client -tool needed is a modern web browser, like safari, firefox, chrome. The -integrated *jupyter notebook* provides a web workspace. By visiting the -workspace, users can do coding, debugging and testing of their programs -online. The **python scipy** series of tools can even display graphical -pictures in the browser. Therefore, it is ideal for data analysis and -processing. +Docklet creates virtual nodes from a base image. Admins can +pre-install development tools and frameworks according to their +interests. The users are also free to install their specific software +in their vcluster. Docklet only need **one** public IP address. The vclusters are configured to use private IP address range, e.g., 172.16.0.0/16, @@ -47,54 +37,34 @@ The Docklet system runtime consists of four components: - docklet master - docklet worker -## install +## Install -Currently the docklet runtime is recommend to run in Unbuntu 15.10+. +Currently the Docklet system is recommend to run in Unbuntu 15.10+. Ensure that python3.5 is the default python3 version. -Unpack the docklet tarball to a directory ( /root/docklet as an -example), will get +Clone Docklet from github ``` -readme.md -prepare.sh -conf/ - container.conf - docklet.conf.template - lxc-script/ -bin/ - docklet-master - docklet-worker -src/ - httprest.py - worker.py - ... -web/ - web.py -doc/ -tools/ - update-basefs.sh - start_jupyter.sh +git clone https://github.com/unias/docklet.git ``` -If it is the first time install, users should run **prepare.sh** to -install necessary packages automatically. Note it may need to run this -script several times to successfully install all the needed packages. +Run **prepare.sh** from console to install depended packages and +generate necessary configurations. -A *root* users will be created for managing the system. The password is -recorded in `FS_PREFIX/local/generated_password.txt` . +A *root* users will be created for managing the Docklet system. The +password is recorded in `FS_PREFIX/local/generated_password.txt` . -## config ## +## Config ## The main configuration file of docklet is conf/docklet.conf. Most default setting works for a single host environment. First copy docklet.conf.template to get docklet.conf. -The following settings should be taken care of: +Pay attention to the following settings: -- NETWORK_DEVICE : the network device to use. +- NETWORK_DEVICE : the network interface to use. - ETCD : the etcd server address. For distributed muli hosts environment, it should be one of the ETCD public server address. For single host environment, the default value should be OK. @@ -111,7 +81,7 @@ The following settings should be taken care of: by visiting this address. If the system is behind a firewall, then a reverse proxy should be setup. -## start ## +## Start ## ### distributed file system ### @@ -123,8 +93,7 @@ Lets presume the file system server export filesystem as nfs In each physical host to run docklet, mount **fileserver:/pub** to **FS_PEFIX/global** . -For single host environment, it need not to configure distributed -file system. +For single host environment, nothing to do. ### etcd ### @@ -133,7 +102,7 @@ Ubuntu releases have included **etcd** in the repository, just `apt-get install etcd`, and it need not to start etcd manually. For others, you should install etcd manually. -For multi hosts distributed environment, start +For multi hosts distributed environment, **must** start **dep/etcd-multi-nodes.sh** in each etcd server hosts. This scripts requires users providing the etcd server address as parameters. @@ -146,27 +115,19 @@ address, e.g., 172.16.0.1. This server will be the master. If it is the first time you start docklet, run `bin/docklet-master init` to init and start docklet master. Otherwise, run `bin/docklet-master start`, which will start master in recovery mode in background using -conf/docklet.conf. It means docklet will recover workspaces existed. - -This script in fact will start three daemons: the docklet master of -httprest.py, the configurable-http-proxy and the docklet web of web.py. +conf/docklet.conf. You can check the daemon status by running `bin/docklet-master status` -If the master failed to start, you could try `bin/docklet-master init` -to initialize the whole system. - -More usages can be found by typing `bin/docklet-master` - The master logs are in **FS_PREFIX/local/log/docklet-master.log** and **docklet-web.log**. ### worker ### -Worker needs a basefs image to boot container. +Worker needs a basefs image to create containers. You can create such an image with `lxc-create -n test -t download`, -and then copy the rootfs to **FS_PREFIX/local**, and renamed `rootfs` +then copy the rootfs to **FS_PREFIX/local**, and renamed `rootfs` to `basefs`. Note the `jupyerhub` package must be installed for this image. And the @@ -179,21 +140,17 @@ Run `bin/docklet-worker start`, will start worker in background. You can check the daemon status by running `bin/docklet-worker status` -More usages can be found by typing `bin/docklet-worker` - The log is in **FS_PREFIX/local/log/docklet-worker.log** Currently, the worker must be run after the master has been started. -## usage ## +## Usage ## Open a browser, visiting the address specified by PORTAL_URL , e.g., ` http://docklet.info/ ` -If the system is just deployed in single host for testing purpose, -then the PORTAL_URL defaults to `http://MASTER_IP:PROXY_PORT`, -e.g., `http://localhost:8000`. - That is it. -## system admin ## +# Contribute # + +Contributions are welcome. Please check [devguide](doc/devguide/devguide.md)