m26508134
/
docklet
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Merge pull request #34 from caodg/updateconf 

update readme
This commit is contained in:
Cao, Donggang 2016-04-08 15:55:25 +08:00
parent 28f577ecb1 d0b8b9e7f5
commit 4b8fc51a57
1 changed files with 40 additions and 83 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

123
README.md
View File

@ -2,38 +2,28 @@
http://docklet.unias.org http://docklet.unias.org
## intro ## Intro
Docklet is an operating system for mini-datacener. Its goal is to help Docklet is a cloud operating system for mini-datacener. Its goal is to
multi-user share cluster resources effectively. Unlike the "application help multi-user share cluster resources effectively. In Docklet, every
framework oriented" cluster manager such as Mesos and Yarn, Docklet is user has their own private **virtual cluster (vcluster)**, which
**user oriented**. In Docklet, every user has their own private consists of a number of virtual Linux container nodes distributed over
**virtual cluster (vcluster)**, which consists of a number of virtual the physical cluster. Each vcluster is separated from others and can be
Linux container nodes distributed over the physical cluster. Every operated like a real physical cluster. Therefore, most applications,
vcluster is separated from others and can be operated like a real especially those requiring a cluster environment, can run in vcluster
physical cluster. Therefore, most applications, especially those seamlessly.
requiring a cluster environment, can run in vcluster seamlessly.
Docklet provides a base image for creating virtual nodes. This image has Users manage and use their vcluster all through web. The only client
pre-installed a lot of mainstream development tools and frameworks, tool needed is a modern web browser supporting HTML5, like Safari,
including gcc/g++, openjdk, python3, R, MPI, scala, ruby, php, node.js, Firefox, or Chrome. The integrated *jupyter notebook* provides a web
texlive, mpich2, spark, **Workspace**. In the Workspace, users can code, debug, test,
scipy/numpy/matplotlib/pandas/sympy/scikit-learn, jupyter notebook, etc. and runn their programs, even visualize the outputs online.
Users can get a ready vcluster with just one click within 1 second. Therefore, it is ideal for data analysis and processing.
The users are free to install their specific software in their vcluster. Docklet creates virtual nodes from a base image. Admins can
Docklet supports operating through **web terminal**. Users can do their pre-install development tools and frameworks according to their
work as an administrator working on a console. The base image system is interests. The users are also free to install their specific software
ubuntu. The recommended way of installing new software is by in their vcluster.
**apt-get**.
The users manage and use their vcluster all through web. The only client
tool needed is a modern web browser, like safari, firefox, chrome. The
integrated *jupyter notebook* provides a web workspace. By visiting the
workspace, users can do coding, debugging and testing of their programs
online. The **python scipy** series of tools can even display graphical
pictures in the browser. Therefore, it is ideal for data analysis and
processing.
Docklet only need **one** public IP address. The vclusters are Docklet only need **one** public IP address. The vclusters are
configured to use private IP address range, e.g., 172.16.0.0/16, configured to use private IP address range, e.g., 172.16.0.0/16,
@ -47,54 +37,34 @@ The Docklet system runtime consists of four components:
- docklet master - docklet master
- docklet worker - docklet worker
## install ## Install
Currently the docklet runtime is recommend to run in Unbuntu 15.10+. Currently the Docklet system is recommend to run in Unbuntu 15.10+.
Ensure that python3.5 is the default python3 version. Ensure that python3.5 is the default python3 version.
Unpack the docklet tarball to a directory ( /root/docklet as an Clone Docklet from github
example), will get
``` ```
readme.md git clone https://github.com/unias/docklet.git
prepare.sh
conf/
container.conf
docklet.conf.template
lxc-script/
bin/
docklet-master
docklet-worker
src/
httprest.py
worker.py
...
web/
web.py
doc/
tools/
update-basefs.sh
start_jupyter.sh
``` ```
If it is the first time install, users should run **prepare.sh** to Run **prepare.sh** from console to install depended packages and
install necessary packages automatically. Note it may need to run this generate necessary configurations.
script several times to successfully install all the needed packages.
A *root* users will be created for managing the system. The password is A *root* users will be created for managing the Docklet system. The
recorded in `FS_PREFIX/local/generated_password.txt` . password is recorded in `FS_PREFIX/local/generated_password.txt` .
## config ## ## Config ##
The main configuration file of docklet is conf/docklet.conf. Most The main configuration file of docklet is conf/docklet.conf. Most
default setting works for a single host environment. default setting works for a single host environment.
First copy docklet.conf.template to get docklet.conf. First copy docklet.conf.template to get docklet.conf.
The following settings should be taken care of: Pay attention to the following settings:
- NETWORK_DEVICE : the network device to use. - NETWORK_DEVICE : the network interface to use.
- ETCD : the etcd server address. For distributed muli hosts - ETCD : the etcd server address. For distributed muli hosts
environment, it should be one of the ETCD public server address. environment, it should be one of the ETCD public server address.
For single host environment, the default value should be OK. For single host environment, the default value should be OK.
@ -111,7 +81,7 @@ The following settings should be taken care of:
by visiting this address. If the system is behind a firewall, then by visiting this address. If the system is behind a firewall, then
a reverse proxy should be setup. a reverse proxy should be setup.
## start ## ## Start ##
### distributed file system ### ### distributed file system ###
@ -123,8 +93,7 @@ Lets presume the file system server export filesystem as nfs
In each physical host to run docklet, mount **fileserver:/pub** to In each physical host to run docklet, mount **fileserver:/pub** to
**FS_PEFIX/global** . **FS_PEFIX/global** .
For single host environment, it need not to configure distributed For single host environment, nothing to do.
file system.
### etcd ### ### etcd ###
@ -133,7 +102,7 @@ Ubuntu releases have included **etcd** in the repository, just `apt-get
install etcd`, and it need not to start etcd manually. For others, you install etcd`, and it need not to start etcd manually. For others, you
should install etcd manually. should install etcd manually.
For multi hosts distributed environment, start For multi hosts distributed environment, **must** start
**dep/etcd-multi-nodes.sh** in each etcd server hosts. This scripts **dep/etcd-multi-nodes.sh** in each etcd server hosts. This scripts
requires users providing the etcd server address as parameters. requires users providing the etcd server address as parameters.
@ -146,27 +115,19 @@ address, e.g., 172.16.0.1. This server will be the master.
If it is the first time you start docklet, run `bin/docklet-master init` If it is the first time you start docklet, run `bin/docklet-master init`
to init and start docklet master. Otherwise, run `bin/docklet-master start`, to init and start docklet master. Otherwise, run `bin/docklet-master start`,
which will start master in recovery mode in background using which will start master in recovery mode in background using
conf/docklet.conf. It means docklet will recover workspaces existed. conf/docklet.conf.
This script in fact will start three daemons: the docklet master of
httprest.py, the configurable-http-proxy and the docklet web of web.py.
You can check the daemon status by running `bin/docklet-master status` You can check the daemon status by running `bin/docklet-master status`
If the master failed to start, you could try `bin/docklet-master init`
to initialize the whole system.
More usages can be found by typing `bin/docklet-master`
The master logs are in **FS_PREFIX/local/log/docklet-master.log** and The master logs are in **FS_PREFIX/local/log/docklet-master.log** and
**docklet-web.log**. **docklet-web.log**.
### worker ### ### worker ###
Worker needs a basefs image to boot container. Worker needs a basefs image to create containers.
You can create such an image with `lxc-create -n test -t download`, You can create such an image with `lxc-create -n test -t download`,
and then copy the rootfs to **FS_PREFIX/local**, and renamed `rootfs` then copy the rootfs to **FS_PREFIX/local**, and renamed `rootfs`
to `basefs`. to `basefs`.
Note the `jupyerhub` package must be installed for this image. And the Note the `jupyerhub` package must be installed for this image. And the
@ -179,21 +140,17 @@ Run `bin/docklet-worker start`, will start worker in background.
You can check the daemon status by running `bin/docklet-worker status` You can check the daemon status by running `bin/docklet-worker status`
More usages can be found by typing `bin/docklet-worker`
The log is in **FS_PREFIX/local/log/docklet-worker.log** The log is in **FS_PREFIX/local/log/docklet-worker.log**
Currently, the worker must be run after the master has been started. Currently, the worker must be run after the master has been started.
## usage ## ## Usage ##
Open a browser, visiting the address specified by PORTAL_URL , Open a browser, visiting the address specified by PORTAL_URL ,
e.g., ` http://docklet.info/ ` e.g., ` http://docklet.info/ `
If the system is just deployed in single host for testing purpose,
then the PORTAL_URL defaults to `http://MASTER_IP:PROXY_PORT`,
e.g., `http://localhost:8000`.
That is it. That is it.
## system admin ## # Contribute #
Contributions are welcome. Please check [devguide](doc/devguide/devguide.md)