Merge branch 'master' of https://github.com/unias/docklet into separate
This commit is contained in:
ooooo
commit
cd0df9847f
|
|
@ -5,3 +5,4 @@ __temp
|
|||
*~
|
||||
.DS_Store
|
||||
docklet.conf
|
||||
home.html
|
||||
|
|
|
|||
123
README.md
123
README.md
|
|
@ -2,38 +2,28 @@
|
|||
|
||||
http://docklet.unias.org
|
||||
|
||||
## intro
|
||||
## Intro
|
||||
|
||||
Docklet is an operating system for mini-datacener. Its goal is to help
|
||||
multi-user share cluster resources effectively. Unlike the "application
|
||||
framework oriented" cluster manager such as Mesos and Yarn, Docklet is
|
||||
**user oriented**. In Docklet, every user has their own private
|
||||
**virtual cluster (vcluster)**, which consists of a number of virtual
|
||||
Linux container nodes distributed over the physical cluster. Every
|
||||
vcluster is separated from others and can be operated like a real
|
||||
physical cluster. Therefore, most applications, especially those
|
||||
requiring a cluster environment, can run in vcluster seamlessly.
|
||||
Docklet is a cloud operating system for mini-datacener. Its goal is to
|
||||
help multi-user share cluster resources effectively. In Docklet, every
|
||||
user has their own private **virtual cluster (vcluster)**, which
|
||||
consists of a number of virtual Linux container nodes distributed over
|
||||
the physical cluster. Each vcluster is separated from others and can be
|
||||
operated like a real physical cluster. Therefore, most applications,
|
||||
especially those requiring a cluster environment, can run in vcluster
|
||||
seamlessly.
|
||||
|
||||
Docklet provides a base image for creating virtual nodes. This image has
|
||||
pre-installed a lot of mainstream development tools and frameworks,
|
||||
including gcc/g++, openjdk, python3, R, MPI, scala, ruby, php, node.js,
|
||||
texlive, mpich2, spark,
|
||||
scipy/numpy/matplotlib/pandas/sympy/scikit-learn, jupyter notebook, etc.
|
||||
Users can get a ready vcluster with just one click within 1 second.
|
||||
Users manage and use their vcluster all through web. The only client
|
||||
tool needed is a modern web browser supporting HTML5, like Safari,
|
||||
Firefox, or Chrome. The integrated *jupyter notebook* provides a web
|
||||
**Workspace**. In the Workspace, users can code, debug, test,
|
||||
and runn their programs, even visualize the outputs online.
|
||||
Therefore, it is ideal for data analysis and processing.
|
||||
|
||||
The users are free to install their specific software in their vcluster.
|
||||
Docklet supports operating through **web terminal**. Users can do their
|
||||
work as an administrator working on a console. The base image system is
|
||||
ubuntu. The recommended way of installing new software is by
|
||||
**apt-get**.
|
||||
|
||||
The users manage and use their vcluster all through web. The only client
|
||||
tool needed is a modern web browser, like safari, firefox, chrome. The
|
||||
integrated *jupyter notebook* provides a web workspace. By visiting the
|
||||
workspace, users can do coding, debugging and testing of their programs
|
||||
online. The **python scipy** series of tools can even display graphical
|
||||
pictures in the browser. Therefore, it is ideal for data analysis and
|
||||
processing.
|
||||
Docklet creates virtual nodes from a base image. Admins can
|
||||
pre-install development tools and frameworks according to their
|
||||
interests. The users are also free to install their specific software
|
||||
in their vcluster.
|
||||
|
||||
Docklet only need **one** public IP address. The vclusters are
|
||||
configured to use private IP address range, e.g., 172.16.0.0/16,
|
||||
|
|
@ -47,54 +37,34 @@ The Docklet system runtime consists of four components:
|
|||
- docklet master
|
||||
- docklet worker
|
||||
|
||||
## install
|
||||
## Install
|
||||
|
||||
Currently the docklet runtime is recommend to run in Unbuntu 15.10+.
|
||||
Currently the Docklet system is recommend to run in Unbuntu 15.10+.
|
||||
|
||||
Ensure that python3.5 is the default python3 version.
|
||||
|
||||
Unpack the docklet tarball to a directory ( /root/docklet as an
|
||||
example), will get
|
||||
Clone Docklet from github
|
||||
|
||||
```
|
||||
readme.md
|
||||
prepare.sh
|
||||
conf/
|
||||
container.conf
|
||||
docklet.conf.template
|
||||
lxc-script/
|
||||
bin/
|
||||
docklet-master
|
||||
docklet-worker
|
||||
src/
|
||||
httprest.py
|
||||
worker.py
|
||||
...
|
||||
web/
|
||||
web.py
|
||||
doc/
|
||||
tools/
|
||||
update-basefs.sh
|
||||
start_jupyter.sh
|
||||
git clone https://github.com/unias/docklet.git
|
||||
```
|
||||
|
||||
If it is the first time install, users should run **prepare.sh** to
|
||||
install necessary packages automatically. Note it may need to run this
|
||||
script several times to successfully install all the needed packages.
|
||||
Run **prepare.sh** from console to install depended packages and
|
||||
generate necessary configurations.
|
||||
|
||||
A *root* users will be created for managing the system. The password is
|
||||
recorded in `FS_PREFIX/local/generated_password.txt` .
|
||||
A *root* users will be created for managing the Docklet system. The
|
||||
password is recorded in `FS_PREFIX/local/generated_password.txt` .
|
||||
|
||||
## config ##
|
||||
## Config ##
|
||||
|
||||
The main configuration file of docklet is conf/docklet.conf. Most
|
||||
default setting works for a single host environment.
|
||||
|
||||
First copy docklet.conf.template to get docklet.conf.
|
||||
|
||||
The following settings should be taken care of:
|
||||
Pay attention to the following settings:
|
||||
|
||||
- NETWORK_DEVICE : the network device to use.
|
||||
- NETWORK_DEVICE : the network interface to use.
|
||||
- ETCD : the etcd server address. For distributed muli hosts
|
||||
environment, it should be one of the ETCD public server address.
|
||||
For single host environment, the default value should be OK.
|
||||
|
|
@ -111,7 +81,7 @@ The following settings should be taken care of:
|
|||
by visiting this address. If the system is behind a firewall, then
|
||||
a reverse proxy should be setup.
|
||||
|
||||
## start ##
|
||||
## Start ##
|
||||
|
||||
### distributed file system ###
|
||||
|
||||
|
|
@ -123,8 +93,7 @@ Lets presume the file system server export filesystem as nfs
|
|||
In each physical host to run docklet, mount **fileserver:/pub** to
|
||||
**FS_PEFIX/global** .
|
||||
|
||||
For single host environment, it need not to configure distributed
|
||||
file system.
|
||||
For single host environment, nothing to do.
|
||||
|
||||
### etcd ###
|
||||
|
||||
|
|
@ -133,7 +102,7 @@ Ubuntu releases have included **etcd** in the repository, just `apt-get
|
|||
install etcd`, and it need not to start etcd manually. For others, you
|
||||
should install etcd manually.
|
||||
|
||||
For multi hosts distributed environment, start
|
||||
For multi hosts distributed environment, **must** start
|
||||
**dep/etcd-multi-nodes.sh** in each etcd server hosts. This scripts
|
||||
requires users providing the etcd server address as parameters.
|
||||
|
||||
|
|
@ -146,27 +115,19 @@ address, e.g., 172.16.0.1. This server will be the master.
|
|||
If it is the first time you start docklet, run `bin/docklet-master init`
|
||||
to init and start docklet master. Otherwise, run `bin/docklet-master start`,
|
||||
which will start master in recovery mode in background using
|
||||
conf/docklet.conf. It means docklet will recover workspaces existed.
|
||||
|
||||
This script in fact will start three daemons: the docklet master of
|
||||
httprest.py, the configurable-http-proxy and the docklet web of web.py.
|
||||
conf/docklet.conf.
|
||||
|
||||
You can check the daemon status by running `bin/docklet-master status`
|
||||
|
||||
If the master failed to start, you could try `bin/docklet-master init`
|
||||
to initialize the whole system.
|
||||
|
||||
More usages can be found by typing `bin/docklet-master`
|
||||
|
||||
The master logs are in **FS_PREFIX/local/log/docklet-master.log** and
|
||||
**docklet-web.log**.
|
||||
|
||||
### worker ###
|
||||
|
||||
Worker needs a basefs image to boot container.
|
||||
Worker needs a basefs image to create containers.
|
||||
|
||||
You can create such an image with `lxc-create -n test -t download`,
|
||||
and then copy the rootfs to **FS_PREFIX/local**, and renamed `rootfs`
|
||||
then copy the rootfs to **FS_PREFIX/local**, and renamed `rootfs`
|
||||
to `basefs`.
|
||||
|
||||
Note the `jupyerhub` package must be installed for this image. And the
|
||||
|
|
@ -179,21 +140,17 @@ Run `bin/docklet-worker start`, will start worker in background.
|
|||
|
||||
You can check the daemon status by running `bin/docklet-worker status`
|
||||
|
||||
More usages can be found by typing `bin/docklet-worker`
|
||||
|
||||
The log is in **FS_PREFIX/local/log/docklet-worker.log**
|
||||
|
||||
Currently, the worker must be run after the master has been started.
|
||||
|
||||
## usage ##
|
||||
## Usage ##
|
||||
|
||||
Open a browser, visiting the address specified by PORTAL_URL ,
|
||||
e.g., ` http://docklet.info/ `
|
||||
|
||||
If the system is just deployed in single host for testing purpose,
|
||||
then the PORTAL_URL defaults to `http://MASTER_IP:PROXY_PORT`,
|
||||
e.g., `http://localhost:8000`.
|
||||
|
||||
That is it.
|
||||
|
||||
## system admin ##
|
||||
# Contribute #
|
||||
|
||||
Contributions are welcome. Please check [devguide](doc/devguide/devguide.md)
|
||||
|
|
|
|||
|
|
@ -76,6 +76,8 @@ pre_start_master () {
|
|||
[ ! -d $FS_PREFIX/global ] && mkdir -p $FS_PREFIX/global
|
||||
[ ! -d $FS_PREFIX/local ] && mkdir -p $FS_PREFIX/local
|
||||
[ ! -d $FS_PREFIX/global/users ] && mkdir -p $FS_PREFIX/global/users
|
||||
[ ! -d $FS_PREFIX/global/sys ] && mkdir -p $FS_PREFIX/global/sys
|
||||
[ ! -d $FS_PREFIX/global/images ] && mkdir -p $FS_PREFIX/global/images
|
||||
[ ! -d $FS_PREFIX/local/volume ] && mkdir -p $FS_PREFIX/local/volume
|
||||
[ ! -d $FS_PREFIX/local/temp ] && mkdir -p $FS_PREFIX/local/temp
|
||||
[ ! -d $FS_PREFIX/local/run ] && mkdir -p $FS_PREFIX/local/run
|
||||
|
|
|
|||
|
|
@ -44,6 +44,7 @@ lxc.cgroup.cpu.cfs_quota_us = %CONTAINER_CPU%
|
|||
lxc.mount.entry = %FS_PREFIX%/global/users/%USERNAME%/data %ROOTFS%/root/nfs none bind,rw,create=dir 0 0
|
||||
lxc.mount.entry = %FS_PREFIX%/global/users/%USERNAME%/hosts/%CLUSTERID%.hosts %ROOTFS%/etc/hosts none bind,ro,create=file 0 0
|
||||
lxc.mount.entry = %FS_PREFIX%/global/users/%USERNAME%/ssh %ROOTFS%/root/.ssh none bind,ro,create=dir 0 0
|
||||
lxc.mount.entry = %FS_PREFIX%/local/temp/%LXCNAME%/ %ROOTFS%/tmp none bind,rw,create=dir 0 0
|
||||
|
||||
# setting hostname
|
||||
lxc.hook.pre-start = HNAME=%HOSTNAME% %LXCSCRIPT%/lxc-prestart
|
||||
|
|
|
|||
|
|
@ -47,16 +47,19 @@
|
|||
# CLUSTER_NET: cluster network ip address range, default is 172.16.0.1/16
|
||||
# CLUSTER_NET=172.16.0.1/16
|
||||
|
||||
# Deprecated since v0.2.7. read from quota group set in web admin page
|
||||
# CONTAINER_CPU: CPU quota of container, default is 100000
|
||||
# A single CPU core has total=100000 (100ms), so the default 100000
|
||||
# mean a single container can occupy a whole core.
|
||||
# For a CPU with two cores, this can be set to 200000
|
||||
# CONTAINER_CPU=100000
|
||||
|
||||
# Deprecated since v0.2.7. read from quota group set in web admin page
|
||||
# CONTAINER_DISK: disk quota of container image upper layer, count in MB,
|
||||
# default is 1000
|
||||
# CONTAINER_DISK=1000
|
||||
|
||||
# Deprecated since v0.2.7. read from quota group set in web admin page
|
||||
# CONTAINER_MEMORY: memory quota of container, count in MB, default is 1000
|
||||
# CONTAINER_MEMORY=1000
|
||||
|
||||
|
|
|
|||
|
|
@ -39,7 +39,9 @@ apt-get install -y etcd
|
|||
which configurable-http-proxy &>/dev/null || npm install -g configurable-http-proxy
|
||||
which configurable-http-proxy &>/dev/null || { echo "Error: install configurable-http-proxy failed, you should try again" && exit 1; }
|
||||
|
||||
echo ""
|
||||
[[ -f conf/docklet.conf ]] || { echo "Generating docklet.conf from template" && cp conf/docklet.conf.template conf/docklet.conf; }
|
||||
[[ -f web/templates/home.html ]] || { echo "Generating HomePage from home.template" && cp web/templates/home.template web/templates/home.html; }
|
||||
|
||||
echo ""
|
||||
echo "All preparation installations are done."
|
||||
|
|
@ -56,7 +58,7 @@ echo "you will get a dicectory structure like"
|
|||
echo " /opt/docklet/local/basefs/etc "
|
||||
echo " /opt/docklet/local/basefs/bin "
|
||||
echo " /opt/docklet/local/basefs/..."
|
||||
echo " "
|
||||
echo "you may want to custom home page of docklet. Please modify web/templates/home.html"
|
||||
|
||||
echo "Next, make sure exim4 can deliver mail out. To enable, run:"
|
||||
echo "dpkg-reconfigure exim4-config"
|
||||
|
|
|
|||
|
|
@ -25,10 +25,11 @@ class Container(object):
|
|||
logger.info("create container %s of %s for %s" %(lxc_name, clustername, username))
|
||||
try:
|
||||
user_info = json.loads(user_info)
|
||||
cpu = user_info["data"]["groupinfo"]["cpu"]
|
||||
cpu = int(user_info["data"]["groupinfo"]["cpu"]) * 100000
|
||||
memory = user_info["data"]["groupinfo"]["memory"]
|
||||
disk = user_info["data"]["groupinfo"]["disk"]
|
||||
image = json.loads(image)
|
||||
status = self.imgmgr.prepareFS(username,image,lxc_name)
|
||||
status = self.imgmgr.prepareFS(username,image,lxc_name,disk)
|
||||
if not status:
|
||||
return [False, "Create container failed when preparing filesystem, possibly insufficient space"]
|
||||
|
||||
|
|
@ -327,8 +328,8 @@ IP=%s
|
|||
onlyglobal.append(container)
|
||||
return [both, onlylocal, onlyglobal]
|
||||
|
||||
def create_image(self,username,imagename,containername,description="not thing",isforce = False):
|
||||
return self.imgmgr.createImage(username,imagename,containername,description,isforce)
|
||||
def create_image(self,username,imagename,containername,description="not thing",imagenum=10):
|
||||
return self.imgmgr.createImage(username,imagename,containername,description,imagenum)
|
||||
|
||||
def flush_container(self,username,imagename,containername):
|
||||
self.imgmgr.flush_one(username,imagename,containername)
|
||||
|
|
|
|||
|
|
@ -23,14 +23,18 @@ class Guest(object):
|
|||
subprocess.getoutput(self.libpath+"/userinit.sh guest")
|
||||
user_info = {}
|
||||
user_info["data"] = {}
|
||||
user_info["data"]["group"] = "primary"
|
||||
user_info["data"]["groupinfo"] = {}
|
||||
user_info["data"]["groupinfo"]["cpu"] = 100000
|
||||
user_info["data"]["groupinfo"]["cpu"] = 4
|
||||
user_info["data"]["groupinfo"]["memory"] = 2000
|
||||
user_info["data"]["groupinfo"]["disk"] = 2000
|
||||
user_info = json.dumps(user_info)
|
||||
self.G_vclustermgr.create_cluster("guestspace", "guest", image, user_info)
|
||||
while True:
|
||||
self.G_vclustermgr.start_cluster("guestspace", "guest")
|
||||
time.sleep(3600)
|
||||
self.G_vclustermgr.stop_cluster("guestspace", "guest")
|
||||
fspath = self.fspath + "/global/local/volume/guest-1-0/"
|
||||
fspath = self.fspath + "/local/volume/guest-1-0/"
|
||||
nfspath = self.fspath + "/global/users/guest/data/"
|
||||
subprocess.getoutput("(cd %s && rm -rf *)" % fspath)
|
||||
subprocess.getoutput("(cd %s && rm -rf *)" % nfspath)
|
||||
|
|
|
|||
|
|
@ -236,17 +236,19 @@ class DockletHttpHandler(http.server.BaseHTTPRequestHandler):
|
|||
description = form.getvalue("description")
|
||||
containername = form.getvalue("containername")
|
||||
isforce = form.getvalue("isforce")
|
||||
if isforce == "true":
|
||||
isforce = True
|
||||
else:
|
||||
isforce = False
|
||||
[status,message] = G_vclustermgr.create_image(user,clustername,containername,imagename,description,isforce)
|
||||
if not isforce == "true":
|
||||
[status,message] = G_vclustermgr.image_check(user,imagename)
|
||||
if not status:
|
||||
self.response(200, {'success':'false','reason':'exists', 'message':message})
|
||||
return [False, "image already exists"]
|
||||
user_info = G_usermgr.selfQuery(cur_user = cur_user)
|
||||
[status,message] = G_vclustermgr.create_image(user,clustername,containername,imagename,description,user_info["data"]["groupinfo"]["image"])
|
||||
if status:
|
||||
logger.info("image has been saved")
|
||||
self.response(200, {'success':'true', 'action':'save'})
|
||||
else:
|
||||
logger.debug(message)
|
||||
self.response(400, {'success':'false', 'message':message})
|
||||
self.response(200, {'success':'false', 'reason':'exceed', 'message':message})
|
||||
|
||||
else:
|
||||
logger.warning ("request not supported ")
|
||||
|
|
@ -344,10 +346,13 @@ class DockletHttpHandler(http.server.BaseHTTPRequestHandler):
|
|||
res['mem_use'] = fetcher.get_mem_use(cmds[2])
|
||||
elif cmds[3] == 'basic_info':
|
||||
res['basic_info'] = fetcher.get_basic_info(cmds[2])
|
||||
user_info = G_usermgr.selfQuery(cur_user = cur_user)
|
||||
self.response(200, {'success':'true', 'monitor':res, 'groupinfo':user_info['data']['groupinfo']})
|
||||
self.response(200, {'success':'true', 'monitor':res})
|
||||
elif cmds[1] == 'user':
|
||||
if not user == 'root':
|
||||
if cmds[2] == 'quotainfo':
|
||||
user_info = G_usermgr.selfQuery(cur_user = cur_user)
|
||||
quotainfo = user_info['data']['groupinfo']
|
||||
self.response(200, {'success':'true', 'quotainfo':quotainfo})
|
||||
'''if not user == 'root':
|
||||
self.response(400, {'success':'false', 'message':'Root Required'})
|
||||
if cmds[3] == 'clustercnt':
|
||||
flag = True
|
||||
|
|
@ -392,7 +397,7 @@ class DockletHttpHandler(http.server.BaseHTTPRequestHandler):
|
|||
else:
|
||||
self.response(200, {'success':'false','message':result})
|
||||
else:
|
||||
self.response(400, {'success':'false', 'message':'not supported request'})
|
||||
self.response(400, {'success':'false', 'message':'not supported request'})'''
|
||||
|
||||
elif cmds[1] == 'listphynodes':
|
||||
res['allnodes'] = G_nodemgr.get_allnodes()
|
||||
|
|
@ -426,6 +431,9 @@ class DockletHttpHandler(http.server.BaseHTTPRequestHandler):
|
|||
elif cmds[1] == 'groupadd':
|
||||
result = G_usermgr.groupadd(form = form, cur_user = cur_user)
|
||||
self.response(200, result)
|
||||
elif cmds[1] == 'quotaadd':
|
||||
result = G_usermgr.quotaadd(form = form, cur_user = cur_user)
|
||||
self.response(200, result)
|
||||
elif cmds[1] == 'groupdel':
|
||||
result = G_usermgr.groupdel(name = form.getvalue('name', None), cur_user = cur_user)
|
||||
self.response(200, result)
|
||||
|
|
|
|||
141
src/imagemgr.py
141
src/imagemgr.py
|
|
@ -25,9 +25,9 @@ import env
|
|||
from lvmtool import *
|
||||
|
||||
class ImageMgr():
|
||||
def sys_call(self,command):
|
||||
output = subprocess.getoutput(command).strip()
|
||||
return None if output == '' else output
|
||||
#def sys_call(self,command):
|
||||
# output = subprocess.getoutput(command).strip()
|
||||
# return None if output == '' else output
|
||||
|
||||
def sys_return(self,command):
|
||||
return_value = subprocess.call(command,shell=True)
|
||||
|
|
@ -58,17 +58,25 @@ class ImageMgr():
|
|||
return self.dealpath(fspath[:-1])
|
||||
else:
|
||||
return fspath
|
||||
|
||||
def createImage(self,user,image,lxc,description="Not thing",isforce = False):
|
||||
|
||||
def createImage(self,user,image,lxc,description="Not thing", imagenum=10):
|
||||
fspath = self.NFS_PREFIX + "/local/volume/" + lxc
|
||||
imgpath = self.imgpath + "private/" + user + "/"
|
||||
if isforce is False:
|
||||
logger.info("this save operation is not force")
|
||||
if os.path.exists(imgpath+image):
|
||||
return [False,"target image is exists"]
|
||||
self.sys_call("mkdir -p %s" % imgpath+image)
|
||||
self.sys_call("rsync -a --delete --exclude=lost+found/ --exclude=nfs/ --exclude=dev/ --exclude=mnt/ --exclude=tmp/ --exclude=media/ --exclude=proc/ --exclude=sys/ %s/ %s/" % (self.dealpath(fspath),imgpath+image))
|
||||
self.sys_call("rm -f %s" % (imgpath+"."+image+"_docklet_share"))
|
||||
|
||||
if not os.path.exists(imgpath+image):
|
||||
cur_imagenum = 0
|
||||
for filename in os.listdir(imgpath):
|
||||
if os.path.isdir(imgpath+filename):
|
||||
cur_imagenum += 1
|
||||
if cur_imagenum >= int(imagenum):
|
||||
return [False,"image number limit exceeded"]
|
||||
try:
|
||||
sys_run("mkdir -p %s" % imgpath+image,True)
|
||||
sys_run("rsync -a --delete --exclude=lost+found/ --exclude=root/nfs/ --exclude=dev/ --exclude=mnt/ --exclude=tmp/ --exclude=media/ --exclude=proc/ --exclude=sys/ %s/ %s/" % (self.dealpath(fspath),imgpath+image),True)
|
||||
sys_run("rm -f %s" % (imgpath+"."+image+"_docklet_share"),True)
|
||||
except Exception as e:
|
||||
logger.error(e)
|
||||
|
||||
self.updateinfo(imgpath,image,description)
|
||||
logger.info("image:%s from LXC:%s create success" % (image,lxc))
|
||||
return [True, "create image success"]
|
||||
|
|
@ -83,7 +91,11 @@ class ImageMgr():
|
|||
imgpath = self.imgpath + "private/" + user + "/"
|
||||
else:
|
||||
imgpath = self.imgpath + "public/" + imageowner + "/"
|
||||
self.sys_call("rsync -a --delete --exclude=lost+found/ --exclude=nfs/ --exclude=dev/ --exclude=mnt/ --exclude=tmp/ --exclude=media/ --exclude=proc/ --exclude=sys/ %s/ %s/" % (imgpath+imagename,self.dealpath(fspath)))
|
||||
try:
|
||||
sys_run("rsync -a --delete --exclude=lost+found/ --exclude=root/nfs/ --exclude=dev/ --exclude=mnt/ --exclude=tmp/ --exclude=media/ --exclude=proc/ --exclude=sys/ %s/ %s/" % (imgpath+imagename,self.dealpath(fspath)),True)
|
||||
except Exception as e:
|
||||
logger.error(e)
|
||||
|
||||
#self.sys_call("rsync -a --delete --exclude=nfs/ %s/ %s/" % (imgpath+image,self.dealpath(fspath)))
|
||||
#self.updatetime(imgpath,image)
|
||||
return
|
||||
|
|
@ -100,8 +112,13 @@ class ImageMgr():
|
|||
if Ret.returncode == 0:
|
||||
logger.info("%s not clean" % layer)
|
||||
sys_run("umount -l %s" % layer)
|
||||
sys_run("rm -rf %s %s" % (rootfs, layer))
|
||||
sys_run("mkdir -p %s %s" % (rootfs, layer))
|
||||
|
||||
try:
|
||||
sys_run("rm -rf %s %s" % (rootfs, layer))
|
||||
sys_run("mkdir -p %s %s" % (rootfs, layer))
|
||||
except Exception as e:
|
||||
logger.error(e)
|
||||
|
||||
|
||||
#prepare volume
|
||||
if check_volume(vgname,lxc):
|
||||
|
|
@ -110,19 +127,18 @@ class ImageMgr():
|
|||
if not new_volume(vgname,lxc,size):
|
||||
logger.error("volume %s create failed" % lxc)
|
||||
return False
|
||||
sys_run("mkfs.ext4 /dev/%s/%s" % (vgname,lxc))
|
||||
sys_run("mount /dev/%s/%s %s" %(vgname,lxc,layer))
|
||||
#self.sys_call("mountpoint %s &>/dev/null && umount -l %s" % (rootfs,rootfs))
|
||||
#self.sys_call("mountpoint %s &>/dev/null && umount -l %s" % (layer,layer))
|
||||
#self.sys_call("rm -rf %s %s && mkdir -p %s %s" % (rootfs,layer,rootfs,layer))
|
||||
#rv = self.sys_return(self.srcpath+"lvmtool.sh check volume %s %s" % (vgname,lxc))
|
||||
#if rv == 1:
|
||||
# self.sys_call(self.srcpath+"lvmtool.sh newvolume %s %s %s %s" % (vgname,lxc,size,layer))
|
||||
#else:
|
||||
# self.sys_call(self.srcpath+"lvmtool.sh mount volume %s %s %s" % (vgname,lxc,layer))
|
||||
#self.sys_call("mkdir -p %s/overlay %s/work" % (layer,layer))
|
||||
#self.sys_call("mount -t overlay overlay -olowerdir=%s/local/basefs,upperdir=%s/overlay,workdir=%s/work %s" % (self.NFS_PREFIX,layer,layer,rootfs))
|
||||
self.sys_call("mount -t aufs -o br=%s=rw:%s/local/basefs=ro+wh none %s/" % (layer,self.NFS_PREFIX,rootfs))
|
||||
|
||||
try:
|
||||
sys_run("mkfs.ext4 /dev/%s/%s" % (vgname,lxc),True)
|
||||
sys_run("mount /dev/%s/%s %s" %(vgname,lxc,layer),True)
|
||||
#self.sys_call("mkdir -p %s/overlay %s/work" % (layer,layer))
|
||||
#self.sys_call("mount -t overlay overlay -olowerdir=%s/local/basefs,upperdir=%s/overlay,workdir=%s/work %s" % (self.NFS_PREFIX,layer,layer,rootfs))
|
||||
sys_run("mount -t aufs -o br=%s=rw:%s/local/basefs=ro+wh none %s/" % (layer,self.NFS_PREFIX,rootfs),True)
|
||||
sys_run("mkdir -p %s/local/temp/%s" % (self.NFS_PREFIX,lxc))
|
||||
|
||||
except Exception as e:
|
||||
logger.error(e)
|
||||
|
||||
logger.info("FS has been prepared for user:%s lxc:%s" % (user,lxc))
|
||||
#self.prepareImage(user,image,layer+"/overlay")
|
||||
self.prepareImage(user,image,layer)
|
||||
|
|
@ -143,7 +159,12 @@ class ImageMgr():
|
|||
sys_run("umount -l %s" % layer)
|
||||
if check_volume(vgname, lxc):
|
||||
delete_volume(vgname, lxc)
|
||||
sys_run("rm -rf %s %s" % (layer,lxcpath))
|
||||
try:
|
||||
sys_run("rm -rf %s %s" % (layer,lxcpath))
|
||||
sys_run("rm -rf %s/local/temp/%s" % (self.NFS_PREFIX,lxc))
|
||||
except Exception as e:
|
||||
logger.error(e)
|
||||
|
||||
return True
|
||||
|
||||
def checkFS(self, lxc, vgname="docklet-group"):
|
||||
|
|
@ -157,15 +178,18 @@ class ImageMgr():
|
|||
sys_run("mount /dev/%s/%s %s" % (vgname,lxc,layer))
|
||||
Ret = sys_run("mountpoint %s" % rootfs)
|
||||
if Ret.returncode != 0:
|
||||
self.sys_call("mount -t aufs -o br=%s=rw:%s/local/basefs=ro+wh none %s/" % (layer,self.NFS_PREFIX,rootfs))
|
||||
sys_run("mount -t aufs -o br=%s=rw:%s/local/basefs=ro+wh none %s/" % (layer,self.NFS_PREFIX,rootfs))
|
||||
return True
|
||||
|
||||
|
||||
def removeImage(self,user,image):
|
||||
imgpath = self.imgpath + "private/" + user + "/"
|
||||
self.sys_call("rm -rf %s/" % imgpath+image)
|
||||
self.sys_call("rm -f %s" % imgpath+"."+image+".info")
|
||||
self.sys_call("rm -f %s" % (imgpath+"."+image+".description"))
|
||||
try:
|
||||
sys_run("rm -rf %s/" % imgpath+image, True)
|
||||
sys_run("rm -f %s" % imgpath+"."+image+".info", True)
|
||||
sys_run("rm -f %s" % (imgpath+"."+image+".description"), True)
|
||||
except Exception as e:
|
||||
logger.error(e)
|
||||
|
||||
def shareImage(self,user,image):
|
||||
imgpath = self.imgpath + "private/" + user + "/"
|
||||
|
|
@ -177,10 +201,13 @@ class ImageMgr():
|
|||
image_info_file = open(imgpath+"."+image+".info", 'w')
|
||||
image_info_file.writelines([createtime, isshare])
|
||||
image_info_file.close()
|
||||
self.sys_call("mkdir -p %s" % (share_imgpath + image))
|
||||
self.sys_call("rsync -a --delete %s/ %s/" % (imgpath+image,share_imgpath+image))
|
||||
self.sys_call("cp %s %s" % (imgpath+"."+image+".info",share_imgpath+"."+image+".info"))
|
||||
self.sys_call("cp %s %s" % (imgpath+"."+image+".description",share_imgpath+"."+image+".description"))
|
||||
try:
|
||||
sys_run("mkdir -p %s" % (share_imgpath + image), True)
|
||||
sys_run("rsync -a --delete %s/ %s/" % (imgpath+image,share_imgpath+image), True)
|
||||
sys_run("cp %s %s" % (imgpath+"."+image+".info",share_imgpath+"."+image+".info"), True)
|
||||
sys_run("cp %s %s" % (imgpath+"."+image+".description",share_imgpath+"."+image+".description"), True)
|
||||
except Exception as e:
|
||||
logger.error(e)
|
||||
|
||||
|
||||
|
||||
|
|
@ -195,10 +222,12 @@ class ImageMgr():
|
|||
image_info_file = open(imgpath+"."+image+".info", 'w')
|
||||
image_info_file.writelines([createtime, isshare])
|
||||
image_info_file.close()
|
||||
self.sys_call("rm -rf %s/" % public_imgpath+image)
|
||||
self.sys_call("rm -f %s" % public_imgpath+"."+image+".info")
|
||||
self.sys_call("rm -f %s" % public_imgpath+"."+image+".description")
|
||||
|
||||
try:
|
||||
sys_run("rm -rf %s/" % public_imgpath+image, True)
|
||||
sys_run("rm -f %s" % public_imgpath+"."+image+".info", True)
|
||||
sys_run("rm -f %s" % public_imgpath+"."+image+".description", True)
|
||||
except Exception as e:
|
||||
logger.error(e)
|
||||
|
||||
def get_image_info(self, user, image, imagetype):
|
||||
if imagetype == "private":
|
||||
|
|
@ -230,9 +259,9 @@ class ImageMgr():
|
|||
images["private"] = []
|
||||
images["public"] = {}
|
||||
imgpath = self.imgpath + "private/" + user + "/"
|
||||
private_images = self.sys_call("ls %s" % imgpath)
|
||||
if private_images is not None and private_images[:3] != "ls:":
|
||||
private_images = private_images.split("\n")
|
||||
try:
|
||||
Ret = sys_run("ls %s" % imgpath, True)
|
||||
private_images = str(Ret.stdout,"utf-8").split()
|
||||
for image in private_images:
|
||||
fimage={}
|
||||
fimage["name"] = image
|
||||
|
|
@ -241,17 +270,18 @@ class ImageMgr():
|
|||
fimage["time"] = time
|
||||
fimage["description"] = description
|
||||
images["private"].append(fimage)
|
||||
else:
|
||||
pass
|
||||
except Exception as e:
|
||||
logger.error(e)
|
||||
|
||||
imgpath = self.imgpath + "public" + "/"
|
||||
public_users = self.sys_call("ls %s" % imgpath)
|
||||
if public_users is not None and public_users[:3] != "ls:":
|
||||
public_users = public_users.split("\n")
|
||||
try:
|
||||
Ret = sys_run("ls %s" % imgpath, True)
|
||||
public_users = str(Ret.stdout,"utf-8").split()
|
||||
for public_user in public_users:
|
||||
imgpath = self.imgpath + "public/" + public_user + "/"
|
||||
public_images = self.sys_call("ls %s" % imgpath)
|
||||
if public_images is not None and public_images[:3] != "ls:":
|
||||
public_images = public_images.split("\n")
|
||||
try:
|
||||
Ret = sys_run("ls %s" % imgpath, True)
|
||||
public_images = str(Ret.stdout,"utf-8").split()
|
||||
images["public"][public_user] = []
|
||||
for image in public_images:
|
||||
fimage = {}
|
||||
|
|
@ -260,8 +290,11 @@ class ImageMgr():
|
|||
fimage["time"] = time
|
||||
fimage["description"] = description
|
||||
images["public"][public_user].append(fimage)
|
||||
else:
|
||||
pass
|
||||
except Exception as e:
|
||||
logger.error(e)
|
||||
except Exception as e:
|
||||
logger.error(e)
|
||||
|
||||
return images
|
||||
|
||||
def isshared(self,user,image):
|
||||
|
|
|
|||
|
|
@ -37,9 +37,12 @@ def new_group(group_name, size = "5000", file_path = "/opt/docklet/local/docklet
|
|||
os.remove(file_path)
|
||||
if not os.path.isdir(file_path[:file_path.rindex("/")]):
|
||||
os.makedirs(file_path[:file_path.rindex("/")])
|
||||
sys_run("dd if=/dev/zero of=%s bs=1M seek=%s count=0" % (file_path,size))
|
||||
sys_run("losetup /dev/loop0 " + file_path)
|
||||
sys_run("vgcreate %s /dev/loop0" % group_name)
|
||||
try:
|
||||
sys_run("dd if=/dev/zero of=%s bs=1M seek=%s count=0" % (file_path,size))
|
||||
sys_run("losetup /dev/loop0 " + file_path)
|
||||
sys_run("vgcreate %s /dev/loop0" % group_name)
|
||||
except Exception as e:
|
||||
logger.error(e)
|
||||
logger.info("initialize lvm group:%s with size %sM success" % (group_name,size))
|
||||
return True
|
||||
|
||||
|
|
@ -55,7 +58,10 @@ def new_group(group_name, size = "5000", file_path = "/opt/docklet/local/docklet
|
|||
Ret = sys_run("vgremove -f " + group_name)
|
||||
if Ret.returncode != 0:
|
||||
logger.error("delete VG %s failed:%s" % (group_name,Ret.stdout.decode('utf-8')))
|
||||
sys_run("vgcreate %s %s" % (group_name,disk))
|
||||
try:
|
||||
sys_run("vgcreate %s %s" % (group_name,disk))
|
||||
except Exception as e:
|
||||
logger.error(e)
|
||||
logger.info("initialize lvm group:%s with size %sM success" % (group_name,size))
|
||||
return True
|
||||
|
||||
|
|
|
|||
|
|
@ -31,8 +31,7 @@ from base64 import b64encode, b64decode
|
|||
import os
|
||||
|
||||
#this class from itsdangerous implements token<->user
|
||||
#from itsdangerous import TimedJSONWebSignatureSerializer as Serializer
|
||||
from itsdangerous import JSONWebSignatureSerializer as Serializer
|
||||
from itsdangerous import TimedJSONWebSignatureSerializer as Serializer
|
||||
from itsdangerous import SignatureExpired, BadSignature
|
||||
|
||||
import env
|
||||
|
|
@ -40,7 +39,7 @@ import env
|
|||
fsdir = env.getenv('FS_PREFIX')
|
||||
|
||||
app = Flask(__name__)
|
||||
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///'+fsdir+'/local/UserTable.db'
|
||||
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///'+fsdir+'/global/sys/UserTable.db'
|
||||
try:
|
||||
secret_key_file = open(env.getenv('FS_PREFIX') + '/local/token_secret_key.txt')
|
||||
app.secret_key = secret_key_file.read()
|
||||
|
|
@ -103,9 +102,8 @@ class User(db.Model):
|
|||
return '<User %r>' % self.username
|
||||
|
||||
#token will expire after 3600s
|
||||
# replace token with no time expiration
|
||||
def generate_auth_token(self, expiration = 3600):
|
||||
s = Serializer(app.config['SECRET_KEY'])
|
||||
s = Serializer(app.config['SECRET_KEY'], expires_in = expiration)
|
||||
str = s.dumps({'id': self.id})
|
||||
return b64encode(str).decode('utf-8')
|
||||
|
||||
|
|
|
|||
|
|
@ -1,23 +1,23 @@
|
|||
#!/usr/bin/python3
|
||||
|
||||
import subprocess,re,sys,etcdlib,psutil
|
||||
import subprocess,re,os,etcdlib,psutil
|
||||
import time,threading,json,traceback,platform
|
||||
|
||||
from log import logger
|
||||
|
||||
class Container_Collector(threading.Thread):
|
||||
|
||||
def __init__(self,etcdaddr,cluster_name,host,cpu_quota,mem_quota,test=False):
|
||||
def __init__(self,etcdaddr,cluster_name,host,test=False):
|
||||
threading.Thread.__init__(self)
|
||||
self.thread_stop = False
|
||||
self.host = host
|
||||
self.etcdser = etcdlib.Client(etcdaddr,"/%s/monitor" % (cluster_name))
|
||||
self.etcdser.setkey('/vnodes/cpu_quota', cpu_quota)
|
||||
self.etcdser.setkey('/vnodes/mem_quota', mem_quota)
|
||||
self.cpu_quota = float(cpu_quota)/100000.0
|
||||
self.mem_quota = float(mem_quota)*1000000/1024
|
||||
self.interval = 2
|
||||
self.test = test
|
||||
self.cpu_last = {}
|
||||
self.cpu_quota = {}
|
||||
self.mem_quota = {}
|
||||
self.cores_num = int(subprocess.getoutput("grep processor /proc/cpuinfo | wc -l"))
|
||||
return
|
||||
|
||||
def list_container(self):
|
||||
|
|
@ -46,22 +46,43 @@ class Container_Collector(threading.Thread):
|
|||
basic_info['PID'] = info['PID']
|
||||
basic_info['IP'] = info['IP']
|
||||
self.etcdser.setkey('/vnodes/%s/basic_info'%(container_name), basic_info)
|
||||
|
||||
cpu_parts = re.split(' +',info['CPU use'])
|
||||
cpu_val = cpu_parts[0].strip()
|
||||
cpu_unit = cpu_parts[1].strip()
|
||||
res = self.etcdser.getkey('/vnodes/%s/cpu_use'%(container_name))
|
||||
cpu_last = 0
|
||||
if res[0] == True:
|
||||
last_use = dict(eval(res[1]))
|
||||
cpu_last = float(last_use['val'])
|
||||
if not container_name in self.cpu_last.keys():
|
||||
confpath = "/var/lib/lxc/%s/config"%(container_name)
|
||||
if os.path.exists(confpath):
|
||||
confile = open(confpath,'r')
|
||||
res = confile.read()
|
||||
lines = re.split('\n',res)
|
||||
for line in lines:
|
||||
words = re.split('=',line)
|
||||
key = words[0].strip()
|
||||
if key == "lxc.cgroup.memory.limit_in_bytes":
|
||||
self.mem_quota[container_name] = float(words[1].strip().strip("M"))*1000000/1024
|
||||
elif key == "lxc.cgroup.cpu.cfs_quota_us":
|
||||
tmp = int(words[1].strip())
|
||||
if tmp == -1:
|
||||
self.cpu_quota[container_name] = self.cores_num
|
||||
else:
|
||||
self.cpu_quota[container_name] = tmp/100000.0
|
||||
quota = {'cpu':self.cpu_quota[container_name],'memory':self.mem_quota[container_name]}
|
||||
self.etcdser.setkey('/vnodes/%s/quota'%(container_name),quota)
|
||||
else:
|
||||
logger.error("Cant't find config file %s"%(confpath))
|
||||
return False
|
||||
self.cpu_last[container_name] = 0
|
||||
cpu_use = {}
|
||||
cpu_use['val'] = cpu_val
|
||||
cpu_use['unit'] = cpu_unit
|
||||
cpu_usedp = (float(cpu_val)-float(cpu_last))/(self.cpu_quota*self.interval*1.3)
|
||||
if(cpu_usedp > 1):
|
||||
cpu_usedp = (float(cpu_val)-float(self.cpu_last[container_name]))/(self.cpu_quota[container_name]*self.interval*1.3)
|
||||
if(cpu_usedp > 1 or cpu_usedp < 0):
|
||||
cpu_usedp = 1
|
||||
cpu_use['usedp'] = cpu_usedp
|
||||
self.cpu_last[container_name] = cpu_val;
|
||||
self.etcdser.setkey('vnodes/%s/cpu_use'%(container_name), cpu_use)
|
||||
|
||||
mem_parts = re.split(' +',info['Memory use'])
|
||||
mem_val = mem_parts[0].strip()
|
||||
mem_unit = mem_parts[1].strip()
|
||||
|
|
@ -70,7 +91,9 @@ class Container_Collector(threading.Thread):
|
|||
mem_use['unit'] = mem_unit
|
||||
if(mem_unit == "MiB"):
|
||||
mem_val = float(mem_val) * 1024
|
||||
mem_usedp = float(mem_val) / self.mem_quota
|
||||
elif (mem_unit == "GiB"):
|
||||
mem_val = float(mem_val) * 1024 * 1024
|
||||
mem_usedp = float(mem_val) / self.mem_quota[container_name]
|
||||
mem_use['usedp'] = mem_usedp
|
||||
self.etcdser.setkey('/vnodes/%s/mem_use'%(container_name), mem_use)
|
||||
#print(output)
|
||||
|
|
@ -220,7 +243,11 @@ class Container_Fetcher:
|
|||
[ret, ans] = self.etcdser.getkey('/%s/cpu_use'%(container_name))
|
||||
if ret == True :
|
||||
res = dict(eval(ans))
|
||||
res['quota'] = self.etcdser.getkey('/cpu_quota')[1]
|
||||
[ret,quota] = self.etcdser.getkey('/%s/quota'%(container_name))
|
||||
if ret == False:
|
||||
res['quota'] = {'cpu':0}
|
||||
logger.warning(quota)
|
||||
res['quota'] = dict(eval(quota))
|
||||
return res
|
||||
else:
|
||||
logger.warning(ans)
|
||||
|
|
@ -231,7 +258,11 @@ class Container_Fetcher:
|
|||
[ret, ans] = self.etcdser.getkey('/%s/mem_use'%(container_name))
|
||||
if ret == True :
|
||||
res = dict(eval(ans))
|
||||
res['quota'] = self.etcdser.getkey('/mem_quota')[1]
|
||||
[ret,quota] = self.etcdser.getkey('/%s/quota'%(container_name))
|
||||
if ret == False:
|
||||
res['quota'] = {'memory':0}
|
||||
logger.warning(quota)
|
||||
res['quota'] = dict(eval(quota))
|
||||
return res
|
||||
else:
|
||||
logger.warning(ans)
|
||||
|
|
|
|||
|
|
@ -247,6 +247,7 @@ class NetworkMgr(object):
|
|||
self.users = {}
|
||||
self.vlanids = {}
|
||||
self.init_vlanids(4095, 60)
|
||||
self.init_shared_vlanids()
|
||||
self.dump_center()
|
||||
self.dump_system()
|
||||
elif mode == 'recovery':
|
||||
|
|
@ -258,6 +259,7 @@ class NetworkMgr(object):
|
|||
self.load_center()
|
||||
self.load_system()
|
||||
self.load_vlanids()
|
||||
self.load_shared_vlanids()
|
||||
else:
|
||||
logger.error("mode: %s not supported" % mode)
|
||||
|
||||
|
|
@ -270,6 +272,17 @@ class NetworkMgr(object):
|
|||
self.vlanids['currentindex'] = i+1
|
||||
self.etcd.setkey("network/vlanids/"+str(i+1), json.dumps(self.vlanids['currentpool']))
|
||||
self.etcd.setkey("network/vlanids/current", str(i+1))
|
||||
|
||||
def init_shared_vlanids(self, vlannum = 128, sharenum = 128):
|
||||
self.shared_vlanids = []
|
||||
for i in range(vlannum):
|
||||
shared_vlanid = {}
|
||||
[status, shared_vlanid['vlanid']] = self.acquire_vlanid()
|
||||
shared_vlanid['sharenum'] = sharenum
|
||||
self.shared_vlanids.append(shared_vlanid)
|
||||
self.etcd.setkey("network/shared_vlanids", json.dumps(self.shared_vlanids))
|
||||
|
||||
|
||||
|
||||
def load_vlanids(self):
|
||||
[status, info] = self.etcd.getkey("network/vlanids/info")
|
||||
|
|
@ -291,6 +304,16 @@ class NetworkMgr(object):
|
|||
pass
|
||||
else:
|
||||
self.etcd.setkey("network/vlanids/"+str(self.vlanids['currentindex']), json.dumps(self.vlanids['currentpool']))
|
||||
|
||||
def load_shared_vlanids(self):
|
||||
[status, shared_vlanids] = self.etcd.getkey("network/shared_vlanids")
|
||||
if not status:
|
||||
self.init_shared_vlanids()
|
||||
else:
|
||||
self.shared_vlanids = json.loads(shared_vlanids)
|
||||
|
||||
def dump_shared_vlanids(self):
|
||||
self.etcd.setkey("network/shared_vlanids", json.dumps(self.shared_vlanids))
|
||||
|
||||
def load_center(self):
|
||||
[status, centerdata] = self.etcd.getkey("network/center")
|
||||
|
|
@ -327,7 +350,18 @@ class NetworkMgr(object):
|
|||
print ("<vlanids>")
|
||||
print (str(self.vlanids['currentindex'])+":"+str(self.vlanids['currentpool']))
|
||||
|
||||
def acquire_vlanid(self):
|
||||
def acquire_vlanid(self, isshared = False):
|
||||
if isshared:
|
||||
if self.shared_vlanids[0]['sharenum'] == 0:
|
||||
self.shared_vlanids.append(self.shared_vlanids.pop(0))
|
||||
if self.shared_vlanids[0]['sharenum'] == 0:
|
||||
logger.info("shared vlanids not enough, add user to full vlanids")
|
||||
for shared_vlanid in self.shared_vlanids:
|
||||
shared_vlanid['sharenum'] = 128
|
||||
self.shared_vlanids[0]['sharenum'] -= 1
|
||||
self.dump_shared_vlanids()
|
||||
return [True, self.shared_vlanids[0]['vlanid']]
|
||||
|
||||
if self.vlanids['currentpool'] == []:
|
||||
if self.vlanids['currentindex'] == 0:
|
||||
return [False, "No VLAN IDs"]
|
||||
|
|
@ -350,7 +384,7 @@ class NetworkMgr(object):
|
|||
self.dump_vlanids()
|
||||
return [True, "Release VLAN ID success"]
|
||||
|
||||
def add_user(self, username, cidr):
|
||||
def add_user(self, username, cidr, isshared = False):
|
||||
logger.info ("add user %s with cidr=%s" % (username, str(cidr)))
|
||||
if self.has_user(username):
|
||||
return [False, "user already exists in users set"]
|
||||
|
|
@ -358,7 +392,7 @@ class NetworkMgr(object):
|
|||
self.dump_center()
|
||||
if status == False:
|
||||
return [False, result]
|
||||
[status, vlanid] = self.acquire_vlanid()
|
||||
[status, vlanid] = self.acquire_vlanid(isshared)
|
||||
if status:
|
||||
vlanid = int(vlanid)
|
||||
else:
|
||||
|
|
@ -372,7 +406,7 @@ class NetworkMgr(object):
|
|||
del self.users[username]
|
||||
return [True, 'add user success']
|
||||
|
||||
def del_user(self, username):
|
||||
def del_user(self, username, isshared = False):
|
||||
logger.info ("delete user %s with cidr=%s" % (username))
|
||||
if not self.has_user(username):
|
||||
return [False, username+" not in users set"]
|
||||
|
|
@ -380,7 +414,8 @@ class NetworkMgr(object):
|
|||
[addr, cidr] = self.users[username].info.split('/')
|
||||
self.center.free(addr, int(cidr))
|
||||
self.dump_center()
|
||||
self.release_vlanid(self.users[username].vlanid)
|
||||
if not isshared:
|
||||
self.release_vlanid(self.users[username].vlanid)
|
||||
netcontrol.del_gw('docklet-br', username)
|
||||
self.etcd.deldir("network/users/"+username)
|
||||
del self.users[username]
|
||||
|
|
|
|||
|
|
@ -20,6 +20,7 @@ from email.mime.multipart import MIMEMultipart
|
|||
from email.header import Header
|
||||
from datetime import datetime
|
||||
import json
|
||||
from log import logger
|
||||
|
||||
email_from_address = env.getenv('EMAIL_FROM_ADDRESS')
|
||||
admin_email_address = env.getenv('ADMIN_EMAIL_ADDRESS')
|
||||
|
|
@ -145,14 +146,27 @@ class userManager:
|
|||
path = env.getenv('DOCKLET_LIB')
|
||||
subprocess.call([path+"/userinit.sh", username])
|
||||
db.session.commit()
|
||||
if not os.path.exists(fspath+"/global/group"):
|
||||
groupfile = open(fspath+"/global/group",'w')
|
||||
if not os.path.exists(fspath+"/global/sys/quota"):
|
||||
groupfile = open(fspath+"/global/sys/quota",'w')
|
||||
groups = []
|
||||
groups.append({'name':'root', 'cpu':'100000', 'memory':'2000', 'imageQuantity':'10', 'lifeCycle':'24'})
|
||||
groups.append({'name':'admin', 'cpu':'100000', 'memory':'2000', 'imageQuantity':'10', 'lifeCycle':'24'})
|
||||
groups.append({'name':'primary', 'cpu':'100000', 'memory':'2000', 'imageQuantity':'10', 'lifeCycle':'24'})
|
||||
groups.append({'name':'root', 'quotas':{ 'cpu':'4', 'disk':'2000', 'memory':'2000', 'image':'10', 'idletime':'24', 'vnode':'8' }})
|
||||
groups.append({'name':'admin', 'quotas':{'cpu':'4', 'disk':'2000', 'memory':'2000', 'image':'10', 'idletime':'24', 'vnode':'8'}})
|
||||
groups.append({'name':'primary', 'quotas':{'cpu':'4', 'disk':'2000', 'memory':'2000', 'image':'10', 'idletime':'24', 'vnode':'8'}})
|
||||
groups.append({'name':'fundation', 'quotas':{'cpu':'4', 'disk':'2000', 'memory':'2000', 'image':'10', 'idletime':'24', 'vnode':'8'}})
|
||||
groupfile.write(json.dumps(groups))
|
||||
groupfile.close()
|
||||
if not os.path.exists(fspath+"/global/sys/quotainfo"):
|
||||
quotafile = open(fspath+"/global/sys/quotainfo",'w')
|
||||
quotas = []
|
||||
quotas.append({'name':'cpu', 'hint':'the cpu quota, number of cores, e.g. 4'})
|
||||
quotas.append({'name':'memory', 'hint':'the memory quota, number of MB , e.g. 4000'})
|
||||
quotas.append({'name':'disk', 'hint':'the disk quota, number of MB, e.g. 4000'})
|
||||
quotas.append({'name':'image', 'hint':'how many images the user can save, e.g. 10'})
|
||||
quotas.append({'name':'idletime', 'hint':'will stop cluster after idletime, number of hours, e.g. 24'})
|
||||
quotas.append({'name':'vnode', 'hint':'how many containers the user can have, e.g. 8'})
|
||||
quotafile.write(json.dumps(quotas))
|
||||
quotafile.close()
|
||||
|
||||
|
||||
def auth_local(self, username, password):
|
||||
password = hashlib.sha512(password.encode('utf-8')).hexdigest()
|
||||
|
|
@ -356,18 +370,18 @@ class userManager:
|
|||
List informantion for oneself
|
||||
'''
|
||||
user = kwargs['cur_user']
|
||||
groupfile = open(fspath+"/global/group",'r')
|
||||
groupfile = open(fspath+"/global/sys/quota",'r')
|
||||
groups = json.loads(groupfile.read())
|
||||
groupfile.close()
|
||||
group = None
|
||||
for one_group in groups:
|
||||
if one_group['name'] == user.user_group:
|
||||
group = one_group
|
||||
group = one_group['quotas']
|
||||
break
|
||||
else:
|
||||
for one_group in groups:
|
||||
if one_group['name'] == "primary":
|
||||
group = one_group
|
||||
group = one_group['quotas']
|
||||
break
|
||||
result = {
|
||||
"success": 'true',
|
||||
|
|
@ -385,12 +399,7 @@ class userManager:
|
|||
"tel" : user.tel,
|
||||
"register_date" : "%s"%(user.register_date),
|
||||
"group" : user.user_group,
|
||||
"groupinfo": {
|
||||
"cpu": group['cpu'],
|
||||
"memory": group['memory'],
|
||||
"imageQuantity": group['imageQuantity'],
|
||||
"lifeCycle":group['lifeCycle'],
|
||||
},
|
||||
"groupinfo": group,
|
||||
},
|
||||
}
|
||||
return result
|
||||
|
|
@ -458,24 +467,17 @@ class userManager:
|
|||
Usage: list(cur_user = token_from_auth)
|
||||
List all groups for an administrator
|
||||
'''
|
||||
groupfile = open(fspath+"/global/group",'r')
|
||||
groupfile = open(fspath+"/global/sys/quota",'r')
|
||||
groups = json.loads(groupfile.read())
|
||||
groupfile.close()
|
||||
quotafile = open(fspath+"/global/sys/quotainfo",'r')
|
||||
quotas = json.loads(quotafile.read())
|
||||
quotafile.close()
|
||||
result = {
|
||||
"success": 'true',
|
||||
"data":[]
|
||||
"groups": groups,
|
||||
"quotas": quotas,
|
||||
}
|
||||
for group in groups:
|
||||
groupinfo = [
|
||||
group['name'],
|
||||
group['cpu'],
|
||||
group['memory'],
|
||||
group['imageQuantity'],
|
||||
group['lifeCycle'],
|
||||
'',
|
||||
]
|
||||
result["data"].append(groupinfo)
|
||||
|
||||
return result
|
||||
|
||||
@administration_required
|
||||
|
|
@ -484,20 +486,14 @@ class userManager:
|
|||
Usage: groupQuery(name = XXX, cur_user = token_from_auth)
|
||||
List a group for an administrator
|
||||
'''
|
||||
groupfile = open(fspath+"/global/group",'r')
|
||||
groupfile = open(fspath+"/global/sys/quota",'r')
|
||||
groups = json.loads(groupfile.read())
|
||||
groupfile.close()
|
||||
for group in groups:
|
||||
if group['name'] == kwargs['name']:
|
||||
result = {
|
||||
"success":'true',
|
||||
"data":{
|
||||
"name" : group['name'] ,
|
||||
"cpu" : group['cpu'] ,
|
||||
"memory" : group['memory'],
|
||||
"imageQuantity" : group['imageQuantity'],
|
||||
"lifeCycle" : group['lifeCycle'],
|
||||
}
|
||||
"data": group,
|
||||
}
|
||||
return result
|
||||
else:
|
||||
|
|
@ -509,7 +505,7 @@ class userManager:
|
|||
Usage: grouplist(cur_user = token_from_auth)
|
||||
List all group names for an administrator
|
||||
'''
|
||||
groupfile = open(fspath+"/global/group",'r')
|
||||
groupfile = open(fspath+"/global/sys/quota",'r')
|
||||
groups = json.loads(groupfile.read())
|
||||
groupfile.close()
|
||||
result = {
|
||||
|
|
@ -524,17 +520,18 @@ class userManager:
|
|||
'''
|
||||
Usage: groupModify(newValue = dict_from_form, cur_user = token_from_auth)
|
||||
'''
|
||||
groupfile = open(fspath+"/global/group",'r')
|
||||
groupfile = open(fspath+"/global/sys/quota",'r')
|
||||
groups = json.loads(groupfile.read())
|
||||
groupfile.close()
|
||||
for group in groups:
|
||||
if group['name'] == kwargs['newValue'].getvalue('groupname',None):
|
||||
form = kwargs['newValue']
|
||||
group['cpu'] = form.getvalue('cpu', '')
|
||||
group['memory'] = form.getvalue('memory', '')
|
||||
group['imageQuantity'] = form.getvalue('image', '')
|
||||
group['lifeCycle'] = form.getvalue('lifecycle', '')
|
||||
groupfile = open(fspath+"/global/group",'w')
|
||||
for key in form.keys():
|
||||
if key == "groupname" or key == "token":
|
||||
pass
|
||||
else:
|
||||
group['quotas'][key] = form.getvalue(key)
|
||||
groupfile = open(fspath+"/global/sys/quota",'w')
|
||||
groupfile.write(json.dumps(groups))
|
||||
groupfile.close()
|
||||
return {"success":'true'}
|
||||
|
|
@ -622,15 +619,52 @@ class userManager:
|
|||
return {"success":'true'}
|
||||
|
||||
@administration_required
|
||||
def groupadd(*args, **kwargs):
|
||||
def quotaadd(*args, **kwargs):
|
||||
form = kwargs.get('form')
|
||||
if (form.getvalue("name") == None):
|
||||
return {"success":'false', "reason": "Empty group name"}
|
||||
groupfile = open(fspath+"/global/group",'r')
|
||||
quotaname = form.getvalue("quotaname")
|
||||
default_value = form.getvalue("default_value")
|
||||
hint = form.getvalue("hint")
|
||||
if (quotaname == None):
|
||||
return { "success":'false', "reason": "Empty quota name"}
|
||||
if (default_value == None):
|
||||
default_value = "--"
|
||||
groupfile = open(fspath+"/global/sys/quota",'r')
|
||||
groups = json.loads(groupfile.read())
|
||||
groupfile.close()
|
||||
groups.append({'name':form.getvalue("name"), 'cpu':form.getvalue("cpu"), 'memory':form.getvalue("memory"), 'imageQuantity':form.getvalue("image"), 'lifeCycle':form.getvalue("lifecycle")})
|
||||
groupfile = open(fspath+"/global/group",'w')
|
||||
for group in groups:
|
||||
group['quotas'][quotaname] = default_value
|
||||
groupfile = open(fspath+"/global/sys/quota",'w')
|
||||
groupfile.write(json.dumps(groups))
|
||||
groupfile.close()
|
||||
quotafile = open(fspath+"/global/sys/quotainfo",'r')
|
||||
quotas = json.loads(quotafile.read())
|
||||
quotafile.close()
|
||||
quotas.append({'name':quotaname, 'hint':hint})
|
||||
quotafile = open(fspath+"/global/sys/quotainfo",'w')
|
||||
quotafile.write(json.dumps(quotas))
|
||||
quotafile.close()
|
||||
return {"success":'true'}
|
||||
|
||||
@administration_required
|
||||
def groupadd(*args, **kwargs):
|
||||
form = kwargs.get('form')
|
||||
groupname = form.getvalue("groupname")
|
||||
if (groupname == None):
|
||||
return {"success":'false', "reason": "Empty group name"}
|
||||
groupfile = open(fspath+"/global/sys/quota",'r')
|
||||
groups = json.loads(groupfile.read())
|
||||
groupfile.close()
|
||||
group = {
|
||||
'name': groupname,
|
||||
'quotas': {}
|
||||
}
|
||||
for key in form.keys():
|
||||
if key == "groupname" or key == "token":
|
||||
pass
|
||||
else:
|
||||
group['quotas'][key] = form.getvalue(key)
|
||||
groups.append(group)
|
||||
groupfile = open(fspath+"/global/sys/quota",'w')
|
||||
groupfile.write(json.dumps(groups))
|
||||
groupfile.close()
|
||||
return {"success":'true'}
|
||||
|
|
@ -640,14 +674,14 @@ class userManager:
|
|||
name = kwargs.get('name', None)
|
||||
if (name == None):
|
||||
return {"success":'false', "reason": "Empty group name"}
|
||||
groupfile = open(fspath+"/global/group",'r')
|
||||
groupfile = open(fspath+"/global/sys/quota",'r')
|
||||
groups = json.loads(groupfile.read())
|
||||
groupfile.close()
|
||||
for group in groups:
|
||||
if group['name'] == name:
|
||||
groups.remove(group)
|
||||
break
|
||||
groupfile = open(fspath+"/global/group",'w')
|
||||
groupfile = open(fspath+"/global/sys/quota",'w')
|
||||
groupfile.write(json.dumps(groups))
|
||||
groupfile.close()
|
||||
return {"success":'true'}
|
||||
|
|
|
|||
|
|
@ -22,3 +22,9 @@ SSH_DIR=$USER_DIR/ssh
|
|||
# maybe it should be delete
|
||||
ssh-keygen -t rsa -P '' -f $SSH_DIR/id_rsa &>/dev/null
|
||||
cp $SSH_DIR/id_rsa.pub $SSH_DIR/authorized_keys
|
||||
|
||||
cat << EOF > $SSH_DIR/config
|
||||
Host *
|
||||
StrictHostKeyChecking no
|
||||
UserKnownHostsFile=/dev/null
|
||||
EOF
|
||||
|
|
|
|||
|
|
@ -59,12 +59,13 @@ class VclusterMgr(object):
|
|||
logger.info ("starting cluster %s with %d containers for %s" % (clustername, int(clustersize), username))
|
||||
workers = self.nodemgr.get_rpcs()
|
||||
image_json = json.dumps(image)
|
||||
groupname = json.loads(user_info)["data"]["group"]
|
||||
if (len(workers) == 0):
|
||||
logger.warning ("no workers to start containers, start cluster failed")
|
||||
return [False, "no workers are running"]
|
||||
# check user IP pool status, should be moved to user init later
|
||||
if not self.networkmgr.has_user(username):
|
||||
self.networkmgr.add_user(username, cidr=29)
|
||||
self.networkmgr.add_user(username, cidr=29, isshared = True if str(groupname) == "fundation" else False)
|
||||
[status, result] = self.networkmgr.acquire_userips_cidr(username, clustersize)
|
||||
gateway = self.networkmgr.get_usergw(username)
|
||||
vlanid = self.networkmgr.get_uservlanid(username)
|
||||
|
|
@ -204,7 +205,14 @@ class VclusterMgr(object):
|
|||
logger.info("flush success")
|
||||
|
||||
|
||||
def create_image(self,username,clustername,containername,imagename,description,isforce=False):
|
||||
def image_check(self,username,imagename):
|
||||
imagepath = self.fspath + "/global/images/private/" + username + "/"
|
||||
if os.path.exists(imagepath + imagename):
|
||||
return [False, "image already exists"]
|
||||
else:
|
||||
return [True, "image not exists"]
|
||||
|
||||
def create_image(self,username,clustername,containername,imagename,description,imagenum=10):
|
||||
[status, info] = self.get_clusterinfo(clustername,username)
|
||||
if not status:
|
||||
return [False, "cluster not found"]
|
||||
|
|
@ -213,7 +221,7 @@ class VclusterMgr(object):
|
|||
if container['containername'] == containername:
|
||||
logger.info("container: %s found" % containername)
|
||||
onework = self.nodemgr.ip_to_rpc(container['host'])
|
||||
res = onework.create_image(username,imagename,containername,description,isforce)
|
||||
res = onework.create_image(username,imagename,containername,description,imagenum)
|
||||
container['lastsave'] = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||||
container['image'] = imagename
|
||||
break
|
||||
|
|
|
|||
|
|
@ -192,7 +192,7 @@ if __name__ == '__main__':
|
|||
logger.info ("using WORKER_PORT %s" % worker_port )
|
||||
|
||||
con_collector = monitor.Container_Collector(etcdaddr, clustername,
|
||||
ipaddr, cpu_quota, mem_quota)
|
||||
ipaddr)
|
||||
con_collector.start()
|
||||
logger.info("CPU and Memory usage monitor started")
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,213 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# 一个R语言实现的爬虫，爬取拉手网美食信息"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"点击下边的cell，点击上方工具栏里的执行图标，即可执行代码块，看到输出结果。代码块左边的In[]出现In[*]表示代码正在执行"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"library(XML)\n",
|
||||
"\n",
|
||||
"giveNames = function(rootNode){\n",
|
||||
" names <- xpathSApply(rootNode,\"//h3/a[@class='goods-name']\",xmlValue)\n",
|
||||
" names\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"givesevices = function(rootNode){\n",
|
||||
" sevices <- xpathSApply(rootNode,\"//h3/a[@class='goods-text']\",xmlValue)\n",
|
||||
" sevices\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"giveprices = function(rootNode){\n",
|
||||
" prices <- xpathSApply(rootNode,\"//div/span[@class='price']\",xmlValue)\n",
|
||||
" prices\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"givemoney = function(rootNode){\n",
|
||||
" money <- xpathSApply(rootNode,\"//div/span[@class='money']\",xmlValue)\n",
|
||||
" money\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"giveplaces = function(rootNode){\n",
|
||||
" places <- xpathSApply(rootNode,\"//a/span[@class='goods-place']\",xmlValue)\n",
|
||||
" places\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"getmeituan = function(URL){\n",
|
||||
" Sys.sleep(runif(1,1,2))\n",
|
||||
" doc<-htmlParse(URL[1],encoding=\"UTF-8\")\n",
|
||||
" rootNode<-xmlRoot(doc)\n",
|
||||
" data.frame(\n",
|
||||
" Names=giveNames(rootNode), #店名\n",
|
||||
" services=givesevices(rootNode), #服务\n",
|
||||
" prices=giveprices(rootNode), #现价\n",
|
||||
" money=givemoney(rootNode), #原价\n",
|
||||
" places=giveplaces(rootNode) #地点\n",
|
||||
" \n",
|
||||
" )\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"URL = paste0(\"http://shenzhen.lashou.com/cate/meishi/page\",1:10)\n",
|
||||
"\n",
|
||||
"mainfunction = function(URL){\n",
|
||||
" data = rbind(\n",
|
||||
" getmeituan (URL[1]),\n",
|
||||
" getmeituan (URL[2]),\n",
|
||||
" getmeituan (URL[3]),\n",
|
||||
" getmeituan (URL[4]),\n",
|
||||
" getmeituan (URL[5])\n",
|
||||
" )\n",
|
||||
" \n",
|
||||
" \n",
|
||||
"}\n",
|
||||
"ll=mainfunction(URL)\n",
|
||||
"write.table(ll,\"result.txt\",row.names=FALSE)\n",
|
||||
"ll\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# R语言的线性回归实例"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"输入数据"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"#读入数据\n",
|
||||
"x<- c(0.10,0.11,0.12,0.13,0.14,0.15,0.16,0.17,0.18,0.20,0.21,0.23)\n",
|
||||
"y<-c(42.0,43.5,45.0,45.5,45.0,47.5,49,53,50,55,55,60)\n",
|
||||
"#绘出 x 与 y 的散列图\n",
|
||||
"plot(y~x)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"执行该段代码，即可看到输出图形；从图中我们可以看出 y 和 x 存在线性相关性，可以进行线性回归分析："
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"model<-lm(y~x)\n",
|
||||
"summary(model)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我们通过 P 值(就是上面的 pr 那一列)来查看对应的解释变量 x 的显著性，通过将 p 值与 0.05 进行比较，若改值小于 0.05，就可以说该变量与被解释变量存在显著的相关性。\n",
|
||||
"\n",
|
||||
"Multiple R-squared 和 Adjusted R-squared 这两个值，就是我们常称为”拟合优度“和”修正的拟合优度“，是指回归方程对样本的拟合程度，这里我们可以看到，修正的拟合优度为 0.9429，表示拟合程度超过五成，这个值越高越好。\n",
|
||||
"\n",
|
||||
"最后，看下 F-statistic，也就是常说的 F 统计量，也称为 F 检验，常用语判断方程整体的显著性实验，其 p 值为 9.505e-08，显然小于 0.05，我们可以认为方程在 P=0.05 的水平上是通过显著性检验的。\n",
|
||||
"\n",
|
||||
"从上面我们看出我们的线性回归效果不错，那么我们可以利用拟合方程进行分类，或者预测。"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"newX<-data.frame(x=0.16)\n",
|
||||
"predict(model,newdata=newX,interval=\"prediction\",level=0.95)#interval=”prediction“ level指定预测的置信区间"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# R语言的逻辑回归示例"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"counts <- c(18,17,15,20,10,20,25,13,12)\n",
|
||||
"outcome <- gl(3,1,9)\n",
|
||||
"treatment <- gl(3,3)\n",
|
||||
"print(d.AD <- data.frame(treatment, outcome, counts))\n",
|
||||
"glm.D93 <- glm(counts ~ outcome + treatment, family = poisson())\n",
|
||||
"anova(glm.D93)\n",
|
||||
"summary(glm.D93)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": []
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "R",
|
||||
"language": "R",
|
||||
"name": "ir"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": "r",
|
||||
"file_extension": ".r",
|
||||
"mimetype": "text/x-r-source",
|
||||
"name": "R",
|
||||
"pygments_lexer": "r",
|
||||
"version": "3.2.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
#!/bin/sh
|
||||
|
||||
# a naive script to fast start spark cluster, assuming host-0 master,
|
||||
# others slaves.
|
||||
# used with dl_stop_spark.sh
|
||||
|
||||
SPARK_HOME=/home/spark
|
||||
|
||||
HOSTS=`grep -v localhost /etc/hosts | awk '{print $2}'`
|
||||
|
||||
echo "Starting master in host-0"
|
||||
|
||||
$SPARK_HOME/sbin/start-master.sh
|
||||
|
||||
for h in $HOSTS ; do
|
||||
echo "Starting slave in $h"
|
||||
if [ $h != 'host-0' ] ; then
|
||||
ssh root@$h /home/spark/sbin/start-slave.sh spark://host-0:7077
|
||||
else
|
||||
/home/spark/sbin/start-slave.sh spark://host-0:7077
|
||||
fi
|
||||
done
|
||||
|
|
@ -0,0 +1,23 @@
|
|||
#!/bin/sh
|
||||
|
||||
# a naive script to stop spark cluster, assuming host-0 master
|
||||
# others slaves
|
||||
# used with dl_start_spark.sh
|
||||
|
||||
SPARK_HOME=/home/spark
|
||||
|
||||
HOSTS=`grep -v localhost /etc/hosts | awk '{print $2}'`
|
||||
|
||||
for h in $HOSTS ; do
|
||||
echo "Stopping slave in $h"
|
||||
if [ $h != 'host-0' ] ; then
|
||||
ssh root@$h /home/spark/sbin/stop-slave.sh
|
||||
else
|
||||
/home/spark/sbin/stop-slave.sh
|
||||
fi
|
||||
done
|
||||
|
||||
echo "Stopping master in host-0"
|
||||
|
||||
$SPARK_HOME/sbin/stop-master.sh
|
||||
|
||||
|
|
@ -0,0 +1,493 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# 用Python分析《美女与野兽》"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"在一篇最近发表的论文*A quantitative analysis of gendered compliments in Disney Princess films*中，Carmen Fought和Karen Eisenhauer发现在这部迪士尼经典影片中女性角色的对话要多于迪士尼近期的电影作品。作者在网络上发现了美女与《野兽》的脚本，因此我立刻用Python重做了他们的分析。\n",
|
||||
"<br />更多地，我在文章最后加入了对《玩具总动员》的分析，这个脚本的形式完全不同，但其中91%的对白来自男性角色。"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"点击下边的cell，点击上方工具栏里的执行图标，即可执行代码块，看到输出结果。代码块左边的In[]出现In[*]表示代码正在执行"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from __future__ import division\n",
|
||||
"\n",
|
||||
"import re\n",
|
||||
"from collections import defaultdict\n",
|
||||
"\n",
|
||||
"import requests\n",
|
||||
"import pandas as pd\n",
|
||||
"import matplotlib\n",
|
||||
"\n",
|
||||
"%matplotlib inline\n",
|
||||
"matplotlib.style.use('ggplot')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Load the script which comes as a text file\n",
|
||||
"\n",
|
||||
"script_url = 'http://www.fpx.de/fp/Disney/Scripts/BeautyAndTheBeast.txt'\n",
|
||||
"script = requests.get(script_url).text"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我们看下脚本的开篇："
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Let's look at the beginning of the script\n",
|
||||
"\n",
|
||||
"script.splitlines()[:20]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"再在中间随意选取一段："
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Let's look at a random place\n",
|
||||
"\n",
|
||||
"script.splitlines()[500:520]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"看上去很容易分析，因为角色和对白间用:隔开"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# seems fairly easy to parse since \n",
|
||||
"# each new speaking line has : and begins with all caps\n",
|
||||
"\n",
|
||||
"def remove_spaces(line):\n",
|
||||
" # remove the weird spaces\n",
|
||||
" return re.sub(' +',' ',line)\n",
|
||||
"\n",
|
||||
"def remove_paren(line):\n",
|
||||
" # remove directions that are not spoken\n",
|
||||
" return re.sub(r'\\([^)]*\\)', '', line)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"lines = []\n",
|
||||
"line = ''\n",
|
||||
"for row in script.splitlines():\n",
|
||||
" if ': ' in row and row[:3].upper() == row[:3]:\n",
|
||||
" line = remove_spaces(line)\n",
|
||||
" line = remove_paren(line)\n",
|
||||
" lines.append(line)\n",
|
||||
" line = row\n",
|
||||
" elif ' ' in row:\n",
|
||||
" line = line + ' ' + row.lstrip()\n",
|
||||
"# don't forget the last line\n",
|
||||
"lines.append(remove_spaces(line))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"lines[:15]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"看看结尾什么样："
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# How does the end look\n",
|
||||
"\n",
|
||||
"lines[-5:]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# 我们去掉可能的空白行\n",
|
||||
"\n",
|
||||
"print (len(lines))\n",
|
||||
"lines = [l for l in lines if len(l) > 0]\n",
|
||||
"print (len(lines))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"现在，我们找出所有角色，并计算他们的出场次数（对白数）"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# now figure out the roles and how many times they appear\n",
|
||||
"\n",
|
||||
"roles = defaultdict(int)\n",
|
||||
"\n",
|
||||
"for line in lines:\n",
|
||||
" # take advantage of the fact that the speaker is always listed before the :\n",
|
||||
" speaker = line.split(':')[0]\n",
|
||||
" roles[speaker] = roles[speaker] + 1"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"len(roles)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"看一下每个角色出现的相对频率："
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# take a look at the relative frequency of each role\n",
|
||||
"roles"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"看起来有一行“to think about”是乱入的（恰好满足了parse条件），我们忽略它"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Looks like there is one bum line ('to think about'')\n",
|
||||
"# But I'll ignore that for now.\n",
|
||||
"\n",
|
||||
"# Quickly eye ball which roles are female and which are possibly mixed groups.\n",
|
||||
"\n",
|
||||
"females = ['WOMAN 1',\n",
|
||||
" 'WOMAN 2',\n",
|
||||
" 'WOMAN 3',\n",
|
||||
" 'WOMAN 4',\n",
|
||||
" 'WOMAN 5',\n",
|
||||
" 'OLD CRONIES',\n",
|
||||
" 'MRS. POTTS',\n",
|
||||
" 'BELLE',\n",
|
||||
" 'BIMBETTE 1'\n",
|
||||
" 'BIMBETTE 2',\n",
|
||||
" 'BIMBETTE 3']\n",
|
||||
"\n",
|
||||
"groups = ['MOB',\n",
|
||||
" 'ALL',\n",
|
||||
" 'BOTH']"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"将每一行对白根据角色性别进行标记，并统计不同性别的对白数量"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Mark each line of dialogue by sex and count them\n",
|
||||
"\n",
|
||||
"sex_lines = {'Male': 0,\n",
|
||||
" 'Female': 0}\n",
|
||||
"\n",
|
||||
"for line in lines:\n",
|
||||
" # Extract speaker \n",
|
||||
" speaker = line.split(':')[0]\n",
|
||||
" \n",
|
||||
" if speaker in females:\n",
|
||||
" sex_lines['Female'] += 1\n",
|
||||
" \n",
|
||||
" elif sex_lines not in groups:\n",
|
||||
" sex_lines['Male'] += 1\n",
|
||||
"\n",
|
||||
"print (sex_lines)\n",
|
||||
"print (sex_lines['Male']/(sex_lines['Male'] + sex_lines['Female']))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"我们使用一张图来显示结果："
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Quick graphical representation \n",
|
||||
"\n",
|
||||
"df = pd.DataFrame([sex_lines.values()],columns=sex_lines.keys())\n",
|
||||
"df.plot(kind='bar')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"也许男性角色和女性角色的对白长度有明显不同？我们来看一看<br/>这次我们计算对白中单词数量而不是计算对白次数："
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Maybe men and women talk for different lengths? This counts words instead of \n",
|
||||
"\n",
|
||||
"sex_words = {'Male': 0,\n",
|
||||
" 'Female': 0}\n",
|
||||
"\n",
|
||||
"for line in lines:\n",
|
||||
" speaker = line.split(':')[0]\n",
|
||||
" dialogue = line.split(':')[1] \n",
|
||||
" # remove the \n",
|
||||
" # tokenize sentence by spaces\n",
|
||||
" word_count = len(dialogue.split(' ')) \n",
|
||||
" \n",
|
||||
" if speaker in females:\n",
|
||||
" sex_words['Female'] += word_count\n",
|
||||
" elif speaker not in groups:\n",
|
||||
" sex_words['Male'] += word_count\n",
|
||||
"\n",
|
||||
"print (sex_words)\n",
|
||||
"print (sex_words['Male']/(sex_words['Male'] + sex_words['Female']))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"也用图表显示出来："
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Quick graphical representation \n",
|
||||
"\n",
|
||||
"df = pd.DataFrame([sex_words.values()],columns=sex_words.keys())\n",
|
||||
"df.plot(kind='bar')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"下面是额外的《玩具总动员》的分析"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Bonus toy story analysis\n",
|
||||
"\n",
|
||||
"url = 'http://www.dailyscript.com/scripts/toy_story.html'\n",
|
||||
"toy_story_script = requests.get(url).text\n",
|
||||
"\n",
|
||||
"# toy_story_script.splitlines()[250:350]\n",
|
||||
"\n",
|
||||
"lines = []\n",
|
||||
"speaker = ''\n",
|
||||
"dialogue = ''\n",
|
||||
"for row in toy_story_script.splitlines()[90:]:\n",
|
||||
" if ' ' in row: \n",
|
||||
" if ':' not in speaker:\n",
|
||||
" lines.append( {'Speaker': remove_paren(speaker).strip(),\n",
|
||||
" 'Dialogue': remove_paren(dialogue).strip() } )\n",
|
||||
" \n",
|
||||
" speaker = remove_spaces(row.strip())\n",
|
||||
" dialogue = ''\n",
|
||||
" elif ' ' in row:\n",
|
||||
" dialogue = dialogue + ' ' + remove_spaces(row)\n",
|
||||
"lines.append( {'Speaker': remove_paren(speaker).strip(),\n",
|
||||
" 'Dialogue': remove_paren(dialogue).strip() } )\n",
|
||||
"\n",
|
||||
"roles = defaultdict(int)\n",
|
||||
"\n",
|
||||
"for line in lines:\n",
|
||||
" speaker = line['Speaker']\n",
|
||||
" roles[speaker] = roles[speaker] + 1\n",
|
||||
"\n",
|
||||
"toy_story_df = pd.DataFrame(lines[1:])\n",
|
||||
"toy_story_df.head()\n",
|
||||
"\n",
|
||||
"toy_story_df.Speaker.value_counts()\n",
|
||||
"\n",
|
||||
"def what_sex(speaker):\n",
|
||||
" if speaker in [\"SID'S MOM\", 'MRS. DAVIS', 'HANNAH', 'BO PEEP']:\n",
|
||||
" return 'Female'\n",
|
||||
" return 'Male'\n",
|
||||
"\n",
|
||||
"toy_story_df['Sex'] = toy_story_df['Speaker'].apply(what_sex)\n",
|
||||
"\n",
|
||||
"sex_df = toy_story_df.groupby('Sex').size()\n",
|
||||
"sex_df.plot(kind='bar')\n",
|
||||
"sex_df\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"def word_count(dialogue):\n",
|
||||
" return len(dialogue.split())\n",
|
||||
"\n",
|
||||
"toy_story_df['Word Count'] = toy_story_df['Dialogue'].apply(word_count)\n",
|
||||
"\n",
|
||||
"word_df = toy_story_df.groupby('Sex')['Word Count'].sum()\n",
|
||||
"word_df.plot(kind='bar')\n",
|
||||
"word_df"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.5.1+"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 0
|
||||
}
|
||||
|
|
@ -72,7 +72,9 @@ echo "[*] Masking dbus.service"
|
|||
chroot $BASEFS systemctl mask dbus.service
|
||||
|
||||
echo "[*] Disabling apache2 service(if installed)"
|
||||
if [ -d $BASEFS/etc/apache2 ] ; then
|
||||
chroot $BASEFS update-rc.d apache2 disable
|
||||
fi
|
||||
|
||||
echo "[*] Disabling ondemand service(if installed)"
|
||||
chroot $BASEFS update-rc.d ondemand disable
|
||||
|
|
@ -81,10 +83,14 @@ echo "[*] Disabling dbus service(if installed)"
|
|||
chroot $BASEFS update-rc.d dbus disable
|
||||
|
||||
echo "[*] Disabling mysql service(if installed)"
|
||||
if [ -d $BASEFS/etc/mysql ] ; then
|
||||
chroot $BASEFS update-rc.d mysql disable
|
||||
fi
|
||||
|
||||
echo "[*] Disabling nginx service(if installed)"
|
||||
if [ -d $BASEFS/etc/nginx ] ; then
|
||||
chroot $BASEFS update-rc.d nginx disable
|
||||
fi
|
||||
|
||||
echo "[*] Setting worker_processes of nginx to 1(if installed)"
|
||||
[ -f $BASEFS/etc/nginx/nginx.conf ] && sed -i -- 's/worker_processes\ auto/worker_processes\ 1/g' $BASEFS/etc/nginx/nginx.conf
|
||||
|
|
@ -104,3 +110,22 @@ cp npmrc $BASEFS/root/.npmrc
|
|||
|
||||
echo "[*] Copying DOCKLET_NOTES.txt to $BASEFS/root/DOCKLET_NOTES.txt"
|
||||
cp DOCKLET_NOTES.txt $BASEFS/root/
|
||||
|
||||
echo "[*] Updating USER/.ssh/config to disable StrictHostKeyChecking"
|
||||
for f in $FS_PREFIX/global/users/* ; do
|
||||
cat <<EOF > $f/ssh/config
|
||||
Host *
|
||||
StrictHostKeyChecking no
|
||||
UserKnownHostsFile=/dev/null
|
||||
EOF
|
||||
done
|
||||
|
||||
echo "[*] Generating $BASEFS/home/spark/sbin/dl_{start|stop}_spark.sh for Spark"
|
||||
if [ -d $BASEFS/home/spark/sbin ] ; then
|
||||
cp dl_*_spark.sh $BASEFS/home/spark/sbin
|
||||
fi
|
||||
|
||||
echo "[*] Generating $BASEFS/root/{R|python}_demo.ipynb"
|
||||
if [ -d $BASEFS/root/ ] ; then
|
||||
cp R_demo.ipynb python_demo.ipynb $BASEFS/root/
|
||||
fi
|
||||
|
|
|
|||
Binary file not shown.
|
Before Width: | Height: | Size: 32 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 480 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 37 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 45 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 50 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 197 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 75 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 27 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 80 KiB |
|
|
@ -7,9 +7,9 @@ function processMemData(data)
|
|||
mem_usedp = data.monitor.mem_use.usedp;
|
||||
var usedp = data.monitor.mem_use.usedp;
|
||||
var unit = data.monitor.mem_use.unit;
|
||||
var quota = data.groupinfo.memory;
|
||||
var quota = data.monitor.mem_use.quota.memory/1024.0;
|
||||
var val = data.monitor.mem_use.val;
|
||||
var out = "("+val+unit+"/"+quota+"MB)";
|
||||
var out = "("+val+unit+"/"+quota.toFixed(2)+"MiB)";
|
||||
$("#con_mem").html((usedp/0.01).toFixed(2)+"%<br/>"+out);
|
||||
}
|
||||
function getMemY()
|
||||
|
|
@ -21,10 +21,9 @@ function processCpuData(data)
|
|||
cpu_usedp = data.monitor.cpu_use.usedp;
|
||||
var val = data.monitor.cpu_use.val;
|
||||
var unit = data.monitor.cpu_use.unit;
|
||||
var quota = data.groupinfo.cpu;
|
||||
quota = quota/1000.0;
|
||||
var quota = data.monitor.cpu_use.quota.cpu;
|
||||
$("#con_cpu").html(val +" "+ unit);
|
||||
$("#con_cpuquota").html(quota.toFixed(2)+"% Cores");
|
||||
$("#con_cpuquota").html(quota + " Cores");
|
||||
}
|
||||
function getCpuY()
|
||||
{
|
||||
|
|
|
|||
|
|
@ -56,7 +56,7 @@
|
|||
<tbody>
|
||||
<tr>
|
||||
<td>base</td>
|
||||
<td><div class="label label-outline-success">public</div></td>
|
||||
<td>public</td>
|
||||
<td>docklet</td>
|
||||
<td>A base image for you</td>
|
||||
<td><div class="i-checks"><label><input type="radio" name="image" value="base_base_base" checked="checked"></label></div></td>
|
||||
|
|
@ -64,7 +64,7 @@
|
|||
{% for image in images['private'] %}
|
||||
<tr>
|
||||
<td>{{image['name']}}</td>
|
||||
<td><div class="label label-outline-warning">{{"private"}}</div></td>
|
||||
<td>private</td>
|
||||
<td>{{user}}</td>
|
||||
<td><a href="/image/description/{{image['name']}}_{{user}}_private/" target="_blank">{{image['description']}}</a></td>
|
||||
<td><div class="i-checks"><label><input type="radio" name="image" value="{{image['name']}}_{{user}}_private"></label></div></td>
|
||||
|
|
@ -74,7 +74,7 @@
|
|||
{% for image in p_images %}
|
||||
<tr>
|
||||
<td>{{image['name']}}</td>
|
||||
<td><div class="label label-outline-success">{{"public"}}</div></td>
|
||||
<td>public</td>
|
||||
<td>{{p_user}}</td>
|
||||
<td><a href="/image/description/{{image['name']}}_{{p_user}}_public" target="_blank">{{image['description']}}</a></td>
|
||||
<td><div class="i-checks"><label><input type="radio" name="image" value="{{image['name']}}_{{p_user}}_public"></label></div></td>
|
||||
|
|
|
|||
|
|
@ -6,7 +6,10 @@
|
|||
{% block panel_list %}
|
||||
<ol class="breadcrumb">
|
||||
<li>
|
||||
<a href="/dashboard/">Home</a>
|
||||
<a href="/dashboard/"><i class="fa fa-dashboard"></i>Home</a>
|
||||
</li>
|
||||
<li class="active">
|
||||
<strong>Admin</strong>
|
||||
</li>
|
||||
</ol>
|
||||
{% endblock %}
|
||||
|
|
@ -48,27 +51,16 @@
|
|||
<div class="modal-body">
|
||||
|
||||
<form action="/group/add/" method="POST" id="addGroupForm">
|
||||
<div class="form-group">
|
||||
<label>Group Name</label>
|
||||
<input type = "text" placeholder="Enter GroupName" class="form-control" name="name" id="mymyname">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>CPU Quota</label>
|
||||
<input type = "text" placeholder="Enter CPU Quota" class="form-control" name="cpu" id="myCpu" value="100000">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>Memory Quota</label>
|
||||
<input type="text" placeholder="Enter Memory Quota" class="form-control" name="memory" id="myMemory" value="2000">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>Image Quantity</label>
|
||||
<input type = "text" placeholder="Enter Image Quantity" class="form-control" name="image" id="myImage" value="10">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>Life Cycle</label>
|
||||
<input type = "text" placeholder="Enter Life Cycle" class="form-control" name="lifecycle" id="myLifecycle" value="24">
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
<label>Name</label>
|
||||
<input type="text" placeholder="Enter Name" class="form-control" name="groupname"/>
|
||||
</div>
|
||||
{% for quota in quotas %}
|
||||
<div class="form-group">
|
||||
<label>{{ quota['name'] }}</label>
|
||||
<input type="text" class="form-control" name={{ quota['name'] }} placeholder="{{quota['hint']}}" />
|
||||
</div>
|
||||
{% endfor %}
|
||||
</form>
|
||||
|
||||
</div>
|
||||
|
|
@ -79,59 +71,97 @@
|
|||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<table id="myGroupTable" class="table table-striped table-bordered">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
<th>CPU</th>
|
||||
<th>Memory</th>
|
||||
<th>ImageQuantity</th>
|
||||
<th>LifeCycle</th>
|
||||
<th>Command</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
</tbody>
|
||||
<div class="modal inmodal" id="ModifyGroupModal" tabindex="-1" role="dialog" aria-hidden="true">
|
||||
<button type="button" class="btn btn-primary btn-sm" data-toggle="modal" data-target="#AddQuotaModal"><i class="fa fa-plus"></i> Add Quota</button>
|
||||
<div class="modal inmodal" id="AddQuotaModal" tabindex="-1" role="dialog" aria-hidden="true">
|
||||
<div class="modal-dialog">
|
||||
<div class="modal-content animated fadeIn">
|
||||
<div class="modal-header">
|
||||
<button type="button" class="close" data-dismiss="modal"><span aria-hidden="true">×</span><span class="sr-only">Close</span></button>
|
||||
<i class="fa fa-laptop modal-icon"></i>
|
||||
<h4 class="modal-title">Modify Group</h4>
|
||||
<small class="font-bold">Modify a group in Docklet</small>
|
||||
<h4 class="modal-title">Add Quota</h4>
|
||||
<small class="font-bold">Add a quota to Docklet</small>
|
||||
</div>
|
||||
<div class="modal-body">
|
||||
<form action="/group/modify/" method="POST" id="modifyGroupForm">
|
||||
<div class="form-group">
|
||||
<label>Group Name</label>
|
||||
<input type = "text" placeholder="Enter Groupname" class="form-control" name="groupname" id="mGroupname" readonly="readonly">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>CPU Quota</label>
|
||||
<input type = "text" placeholder="Enter CPU Quota" class="form-control" name="cpu" id="mCpu">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>Memory Quota</label>
|
||||
<input type="text" placeholder="Enter Memory Quota" class="form-control" name="memory" id="mMemory">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>Image Quantity</label>
|
||||
<input type = "text" placeholder="Enter Image Quantity" class="form-control" name="image" id="mImage">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>Life Cycle</label>
|
||||
<input type = "text" placeholder="Enter Life Cycle" class="form-control" name="lifecycle" id="mLifecycle">
|
||||
</div>
|
||||
</form>
|
||||
|
||||
<form action="/quota/add/" method="POST" id="addQuotaForm">
|
||||
<div class="form-group">
|
||||
<label>Name</label>
|
||||
<input type="text" placeholder="Enter Name" class="form-control" name="quotaname"/>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>Default Value</label>
|
||||
<input type="text" placeholder="Enter Default Value" class="form-control" name="default_value"/>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>Hint</label>
|
||||
<input type="text" placeholder="Enter Hint" class="form-control" name="hint"/>
|
||||
</div>
|
||||
</form>
|
||||
|
||||
</div>
|
||||
<div class="modal-footer">
|
||||
<button type="button" class="btn btn-white" data-dismiss="modal">Close</button>
|
||||
<button type="button" class="btn btn-primary" onClick="javascript:sendModifyGroup();">Submit</button>
|
||||
<button type="button" class="btn btn-primary" onClick="javascript:sendAddQuota();">Submit</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<table id="myGroupTable" class="table table-striped table-bordered">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
{% for quota in quotas %}
|
||||
<th> {{ quota['name'] }} </th>
|
||||
{% endfor %}
|
||||
<th>Command</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{% for group in groups %}
|
||||
<tr>
|
||||
<th>{{ group['name'] }}</th>
|
||||
{% for quota in quotas %}
|
||||
<th> {{ group['quotas'][quota['name']] }} </th>
|
||||
{% endfor %}
|
||||
<th><a class="btn btn-xs btn-info" data-toggle="modal" data-target="#ModifyGroupModal_{{ group['name'] }}">Edit</a>
|
||||
{% if group['name'] in [ "root", "primary", "admin", "fundation" ] %}
|
||||
<a class="btn btn-xs btn-default" href="javascript:void(0)">Delete</a></th>
|
||||
{% else %}
|
||||
<a class="btn btn-xs btn-danger" href="/group/delete/{{group['name']}}">Delete</a></th>
|
||||
{% endif %}
|
||||
<div class="modal inmodal" id="ModifyGroupModal_{{ group['name'] }}" tabindex="-1" role="dialog" aria-hidden="true">
|
||||
<div class="modal-dialog">
|
||||
<div class="modal-content animated fadeIn">
|
||||
<div class="modal-header">
|
||||
<button type="button" class="close" data-dismiss="modal"><span aria-hidden="true">×</span><span class="sr-only">Close</span></button>
|
||||
<i class="fa fa-laptop modal-icon"></i>
|
||||
<h4 class="modal-title">Modify Group</h4>
|
||||
<small class="font-bold">Modify a group in Docklet</small>
|
||||
</div>
|
||||
<form action="/group/modify/{{group['name']}}/" method="POST" >
|
||||
<div class="modal-body">
|
||||
<div class="form-group">
|
||||
<label>Name</label>
|
||||
<input type="text" placeholder="Enter Name" class="form-control" name="groupname" readonly="true" value={{ group['name'] }} />
|
||||
</div>
|
||||
{% for quota in quotas %}
|
||||
<div class="form-group">
|
||||
<label> {{ quota['name'] }}</label>
|
||||
<input type="text" placeholder="{{ quota['hint'] }}" class="form-control" name={{ quota['name'] }} value={{ group['quotas'][quota['name']] }} />
|
||||
</div>
|
||||
{% endfor %}
|
||||
</div>
|
||||
<div class="modal-footer">
|
||||
<button type="button" class="btn btn-white" data-dismiss="modal">Close</button>
|
||||
<button type="submit" class="btn btn-primary">Submit</button>
|
||||
</div>
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</tr>
|
||||
{% endfor %}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
|
|
@ -146,30 +176,11 @@
|
|||
|
||||
|
||||
<script type="text/javascript">
|
||||
$(document).ready(function() {
|
||||
var gTable = $('#myGroupTable').dataTable({
|
||||
"ajax": {
|
||||
"url": "/group/detail/",
|
||||
"type": "POST"
|
||||
},
|
||||
//"scrollX": true,
|
||||
"columnDefs": [
|
||||
{
|
||||
"render": function ( data, type, row ) {
|
||||
return '<a class="btn btn-info btn-sm" data-toggle="modal" data-target="#ModifyGroupModal" onClick="javascript:setFormGroup('+ "'" + row[0] + "'" + ');">' + 'Edit' + '</a>'
|
||||
+ '<a class="btn btn-danger btn-sm" href="/group/delete/' + row[0] +'">' + 'Delete' + '</a>';
|
||||
},
|
||||
"targets": 5
|
||||
},
|
||||
]
|
||||
|
||||
});
|
||||
});
|
||||
function sendAddGroup(){
|
||||
document.getElementById("addGroupForm").submit();
|
||||
}
|
||||
function sendModifyGroup(){
|
||||
document.getElementById("modifyGroupForm").submit();
|
||||
function sendAddQuota(){
|
||||
document.getElementById("addQuotaForm").submit();
|
||||
}
|
||||
function setFormGroup(arg){
|
||||
$.post("/group/query/",
|
||||
|
|
|
|||
|
|
@ -16,6 +16,9 @@
|
|||
<li>
|
||||
<a href="/dashboard/"><i class="fa fa-dashboard"></i>Home</a>
|
||||
</li>
|
||||
<li class="active">
|
||||
<strong>Config</strong>
|
||||
</li>
|
||||
</ol>
|
||||
{% endblock %}
|
||||
|
||||
|
|
@ -65,7 +68,7 @@
|
|||
</div>
|
||||
<div class="modal-body">
|
||||
<div class="form-group">
|
||||
<form action="/workspace/scaleout/{{ clustername }}/" method="POST" id="scaleout">
|
||||
<form action="/workspace/scaleout/{{ clustername }}/" method="POST" >
|
||||
<table class="table table-striped table-bordered table-hover table-image">
|
||||
<thead>
|
||||
<tr>
|
||||
|
|
@ -78,14 +81,14 @@
|
|||
<tbody>
|
||||
<tr>
|
||||
<td>base</td>
|
||||
<td><div class="label label-outline-success">public</div></td>
|
||||
<td>public</td>
|
||||
<td>docklet</td>
|
||||
<td><input type="radio" name="image" value="base_base_base" checked="checked"></td>
|
||||
</tr>
|
||||
{% for image in images['private'] %}
|
||||
<tr>
|
||||
<td>{{image['name']}}</td>
|
||||
<td><div class="label label-outline-warning">private</div></td>
|
||||
<td>private</td>
|
||||
<td>{{mysession['username']}}</td>
|
||||
<td><input type="radio" name="image" value="{{image['name']}}_{{mysession['username']}}_private"></td>
|
||||
</tr>
|
||||
|
|
@ -94,7 +97,7 @@
|
|||
{% for image in p_images %}
|
||||
<tr>
|
||||
<td>{{image['name']}}</td>
|
||||
<td><div class="label label-outline-success">public</div></td>
|
||||
<td>public</td>
|
||||
<td>{{p_user}}</td>
|
||||
<td><input type="radio" name="image" value="{{image['name']}}_{{p_user}}_public"></td>
|
||||
</tr>
|
||||
|
|
@ -244,7 +247,7 @@
|
|||
<tbody>
|
||||
<tr>
|
||||
<td>base</td>
|
||||
<td><div class="label label-outline-success">public</div></td>
|
||||
<td>public</td>
|
||||
<td>docklet</td>
|
||||
<td>2015-01-01 00:00:00</td>
|
||||
<td>A Base Image For You</td>
|
||||
|
|
@ -254,18 +257,18 @@
|
|||
{% for image in images['private'] %}
|
||||
<tr>
|
||||
<td>{{image['name']}}</td>
|
||||
<td><div class="label label-outline-warning">{{"private"}}</div></td>
|
||||
<td>private</td>
|
||||
<td>{{mysession['username']}}</td>
|
||||
<td>{{image['time']}}</td>
|
||||
<td><a href="/image/description/{{image['name']}}_{{mysession['username']}}_private/" target="_blank">{{image['description']}}</a></td>
|
||||
{% if image['isshared'] == 'false' %}
|
||||
<td><div class="label label-outline-default">unshared</div></td>
|
||||
<td>unshared</td>
|
||||
<td>
|
||||
<a href="/image/share/{{ image['name'] }}/"><button type="button" class="btn btn-xs btn-success">share</button></a>
|
||||
<a href="/image/delete/{{ image['name'] }}/"><button type="button" class="btn btn-xs btn-danger">delete</button></a>
|
||||
</td>
|
||||
{% else %}
|
||||
<td><div class="label label-outline-default">shared</div></td>
|
||||
<td>shared</td>
|
||||
<td>
|
||||
<a href="/image/unshare/{{ image['name'] }}/"><button type="button" class="btn btn-xs btn-warning">unshare</button></a>
|
||||
<a href="/image/delete/{{ image['name'] }}/"><button type="button" class="btn btn-xs btn-danger">delete</button></a>
|
||||
|
|
@ -277,7 +280,7 @@
|
|||
{% for image in p_images %}
|
||||
<tr>
|
||||
<td>{{image['name']}}</td>
|
||||
<td><div class="label label-outline-success">{{"public"}}</div></td>
|
||||
<td>public</td>
|
||||
<td>{{p_user}}</td>
|
||||
<td>{{image['time']}}</td>
|
||||
<td><a href="/image/description/{{image['name']}}_{{p_user}}_public/" target="_blank">{{image['description']}}</a></td>
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@
|
|||
<a href="/dashboard/"><i class="fa fa-dashboard"></i>Home</a>
|
||||
</li>
|
||||
<li class="active">
|
||||
<a href='/dashboard/'>Dashboard</a>
|
||||
<strong>Dashboard</strong>
|
||||
</li>
|
||||
</ol>
|
||||
{% endblock %}
|
||||
|
|
@ -56,7 +56,7 @@
|
|||
<a href="/go/{{ mysession['username'] }}/{{ cluster['name'] }}" target="_blank"><button type="button" class="btn btn-xs btn-success"> Go </button></a>
|
||||
</td>
|
||||
{% else %}
|
||||
<td><a href="/monitor/Node/"><div class="text-warning"><i class="fa fa-stop "></i> Stopped</div></a></td>
|
||||
<td><a href="/vclusters/"><div class="text-warning"><i class="fa fa-stop "></i> Stopped</div></a></td>
|
||||
<td>
|
||||
<a href="/workspace/start/{{ cluster['name'] }}/"><button type="button" class="btn btn-xs btn-success"> Start </button></a>
|
||||
<a href="/workspace/delete/{{ cluster['name'] }}/"><button type="button" class="btn btn-xs btn-danger">Delete</button></a>
|
||||
|
|
|
|||
|
|
@ -19,11 +19,10 @@
|
|||
<h2 class="headline text-red">500</h2>
|
||||
|
||||
<div class="error-content">
|
||||
<h3><br/><i class="fa fa-warning text-red"></i> Internal Server Error</h3>
|
||||
<h3><br/><i class="fa fa-warning text-red"></i> {{ title }}</h3>
|
||||
|
||||
<p>
|
||||
The server encountered something unexpected that didn't allow it to complete the request. We apologize.You can go back to
|
||||
<a href="/dashboard/">dashboard</a> or <a href="/logout">log out</a>
|
||||
{{reason|safe}}
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
|
|
|||
|
|
@ -1,113 +1,139 @@
|
|||
<!DOCTYPE html>
|
||||
<html>
|
||||
|
||||
<head>
|
||||
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
|
||||
<title>Docklet | Home</title>
|
||||
<link rel="shortcut icon" href="/static/img/favicon.ico">
|
||||
|
||||
<link href="http://cdn.bootcss.com/bootstrap/3.3.5/css/bootstrap.min.css" rel="stylesheet">
|
||||
<link href="http://cdn.bootcss.com/font-awesome/4.3.0/css/font-awesome.min.css" rel="stylesheet">
|
||||
<link href="http://cdn.bootcss.com/animate.css/2.0/animate.min.css" rel="stylesheet">
|
||||
<link href="/static/css/docklet.css" rel="stylesheet">
|
||||
<style type="text/css">
|
||||
h1 { font-weight: 400 }
|
||||
h2 { font-weight: 300 }
|
||||
.underline { border-bottom : 1px solid #FFFFFF }
|
||||
a:link {text-decoration: none }
|
||||
</style>
|
||||
|
||||
</head>
|
||||
|
||||
<body style="background-color:#FFFFFF">
|
||||
|
||||
<div class="navbar navbar-fixed-top" role="navigation" style="border:none !important; background: #F3F3F3; opacity:0.9">
|
||||
<div class="container">
|
||||
<div class="row" style="padding-top:8px; padding-bottom:5px;">
|
||||
<div class="col-lg-2 col-md-2 col-sm-4 col-xs-4">
|
||||
<img src="/static/img/logoname.png" style="width:100%">
|
||||
</div>
|
||||
|
||||
<div class="col-lg-1 col-lg-offset-8 col-md-1 col-md-offset-8 col-sm-2 col-sm-offset-4 col-xs-4 col-xs-offset-0" style="padding-left:0px">
|
||||
<a href="/login/"><button type="button" class="btn btn-outline-success btn-block">Login</button></a>
|
||||
</div>
|
||||
<div class="col-lg-1 col-md-1 col-sm-2 col-xs-4" style="padding-left:0px">
|
||||
<a href="/dashboard_guest/"><button type="button" class="btn btn-outline-success btn-block">Try</button></a>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
<!-- <div class="container" style="width:100%"> -->
|
||||
<div class="container">
|
||||
<div class="row" style="margin-top: 100px; margin-bottom:50px; margin-left:0px; margin-right:0px; background: #FFFFFF">
|
||||
<img src="/static/img/home.png" style="width:100%">
|
||||
</div>
|
||||
|
||||
<div class="row docklet-red-block" style="padding-top:100px; padding-bottom:100px">
|
||||
<div class="col-lg-3 col-lg-offset-1 col-md-3 col-md-offset-1 col-sm-4 col-sm-offset-0 col-xs-8 col-xs-offset-2">
|
||||
<img src="/static/img/workspace.png" style="width:100%">
|
||||
</div>
|
||||
<div class="col-lg-6 col-lg-offset-1 col-md-6 col-md-offset-1 col-sm-8 col-sm-offset-0 col-xs-12 col-xs-offset-0">
|
||||
<h1>Workspace = Cluster+Service+Data</h1>
|
||||
<h2>Package service and data based on virtual cluster as virtual compute environment for your work. This is your Workspace !</h2>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="row docklet-green-block" style="padding-top:100px; padding-bottom:100px">
|
||||
<div class="col-lg-6 col-lg-offset-1 col-md-6 col-md-offset-1 col-sm-8 col-sm-offset-0 col-xs-12 col-xs-offset-0">
|
||||
<h1>Click and Go</h1>
|
||||
<h2>Distributed or single node ? Never mind !
|
||||
Click it just like start an app on your smart phone, and your workspace is
|
||||
ready for you.</h2>
|
||||
</div>
|
||||
<div class="col-lg-3 col-lg-offset-1 col-md-3 col-md-offset-1 col-sm-4 col-sm-offset-0 col-xs-8 col-xs-offset-2">
|
||||
<img src="/static/img/app.png" style="width:100%">
|
||||
</div>
|
||||
|
||||
</div>
|
||||
<div class="row docklet-yellow-block" style="padding-top:100px; padding-bottom:100px">
|
||||
<div class="col-lg-3 col-lg-offset-1 col-md-3 col-md-offset-1 col-sm-4 col-sm-offset-0 col-xs-8 col-xs-offset-2">
|
||||
<img src="/static/img/web.png" style="width:100%">
|
||||
</div>
|
||||
<div class="col-lg-6 col-lg-offset-1 col-md-6 col-md-offset-1 col-sm-8 col-sm-offset-0 col-xs-12 col-xs-offset-0">
|
||||
<h1>All in Web</h1>
|
||||
<h2>All you need is a web browser.
|
||||
Compute in web, code in web, plot in web, anything in web !
|
||||
You can get to work anytime and anywhere by internet.</h2>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
<div class="row docklet-blue-block" style="padding-top:100px; padding-bottom:100px">
|
||||
<div class="col-lg-8 col-lg-offset-2 col-md-10 col-md-offset-1 col-sm-10 col-sm-offset-1 col-xs-12 col-xs-offset-0">
|
||||
<h2>Now,   jupyter / python3 / matplotlib / sklearn /scipy / numpy / pandas / latex is ready for you</h2>
|
||||
<h2>And,   more workspaces are coming for your <span class="underline">data processing</span> / <span class="underline">data mining</span> / <span class="underline">machine learning</span> work</h2>
|
||||
<br/>
|
||||
<a href="/login/"><button type="button" class="btn btn-lg btn-outline-warning">Get to Started</button></a>
|
||||
<span>   </span>
|
||||
<a href="/dashboard_guest/"><button type="button" class="btn btn-lg btn-outline-warning">Have a Try</button></a>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
|
||||
<div class="row">
|
||||
<p class="m-t"> <small>Copyright© 2016 <a href="http://docklet.unias.org">UniAS</a>@<a href="http://www.sei.pku.edu.cn"> SEI, PKU</a></small> </p>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
|
||||
<!-- Mainly scripts -->
|
||||
<script src="http://cdn.bootcss.com/jquery/2.2.1/jquery.min.js"></script>
|
||||
<script src="http://cdn.bootcss.com/bootstrap/3.3.5/js/bootstrap.min.js"></script>
|
||||
|
||||
</body>
|
||||
|
||||
</html>
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
|
||||
<title>Docklet | Home</title>
|
||||
<link rel="shortcut icon" href="/static/img/favicon.ico">
|
||||
|
||||
<link href="http://cdn.bootcss.com/bootstrap/3.3.5/css/bootstrap.min.css" rel="stylesheet">
|
||||
<link href="http://cdn.bootcss.com/font-awesome/4.5.0/css/font-awesome.css" rel="stylesheet">
|
||||
|
||||
<style type="text/css">
|
||||
a.linkbtn, a.linkbtn:visited, a.linkbtn:active{
|
||||
color:white;
|
||||
padding:8px;
|
||||
}
|
||||
|
||||
a.linkbtn:hover{
|
||||
color:white;
|
||||
text-decoration:none;
|
||||
border-bottom: 1px solid white;
|
||||
|
||||
}
|
||||
.navbar-custom-top{
|
||||
background:transparent;
|
||||
position:absolute;
|
||||
z-index:1030;
|
||||
left:0px;
|
||||
right:0px;
|
||||
}
|
||||
</style>
|
||||
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<div class="navbar navbar-custom-top" role="navigation">
|
||||
<div class="container">
|
||||
<div class="row" style="font-size:16px; color:white; padding:16px">
|
||||
<div class="pull-right" >
|
||||
<a class="linkbtn" href="http://docklet.unias.org/docklet-book/userguide/_book/">Document</a>
|
||||
·
|
||||
<a class="linkbtn" href="/login/" >Sign In</a>
|
||||
</div>
|
||||
<div>
|
||||
<a class="linkbtn" href="http://docklet.unias.org"><strong>Docklet Cloud OS</strong></a>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Carousel -->
|
||||
<div id="myCarousel" class="carousel slide" data-ride="carousel" style="box-shadow:0px 2px 10px 0px black">
|
||||
<!-- Indicators -->
|
||||
<ol class="carousel-indicators">
|
||||
<li data-target="#myCarousel" data-slide-to="0" class="active"></li>
|
||||
<li data-target="#myCarousel" data-slide-to="1"></li>
|
||||
</ol>
|
||||
<div class="carousel-inner" role="listbox">
|
||||
<div class="item active">
|
||||
<img src="/static/img/home/cloud.png" alt="Cloud OS" style="min-height:600px">
|
||||
<div class="container">
|
||||
<div class="carousel-caption">
|
||||
<h1>Cloud OS</h1>
|
||||
<p class="lead">Cloud OS for your cloud Apps.</p>
|
||||
<br>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="item">
|
||||
<img src="/static/img/home/workspace.png" alt="Cloud Workspace" style="min-height:600px">
|
||||
<div class="container">
|
||||
<div class="carousel-caption">
|
||||
<h1>Cloud Workspace</h1>
|
||||
<p class="lead">Workspace in cloud, all your work in cloud.</p>
|
||||
<br>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<a class="left carousel-control" href="#myCarousel" role="button" data-slide="prev">
|
||||
<span class="glyphicon glyphicon-chevron-left" aria-hidden="true"></span>
|
||||
<span class="sr-only">Previous</span>
|
||||
</a>
|
||||
<a class="right carousel-control" href="#myCarousel" role="button" data-slide="next">
|
||||
<span class="glyphicon glyphicon-chevron-right" aria-hidden="true"></span>
|
||||
<span class="sr-only">Next</span>
|
||||
</a>
|
||||
</div><!-- /.carousel -->
|
||||
|
||||
<div class="container" style="margin-top:60px">
|
||||
<div class="row">
|
||||
<div class="col-lg-6 col-lg-offset-1 col-md-7 col-md-offset-0 col-sm-7 col-sm-offset-0 col-xs-12 col-xs-offset-0">
|
||||
<h2>Workspace=Cluster+Service+Data</h2>
|
||||
<br/>
|
||||
<p class="lead">Package service and data based on virtual cluster as virtual compute environment for your work.
|
||||
<br> This is your Workspace !</p>
|
||||
</div>
|
||||
<div class="col-lg-3 col-lg-offset-1 col-md-4 col-md-offset-1 col-sm-5 col-sm-offset-0 col-xs-10 col-xs-offset-1">
|
||||
<img src="/static/img/home/app-workspace.png" alt="feature-workspace" width="100%">
|
||||
</div>
|
||||
</div>
|
||||
<hr>
|
||||
<div class="row">
|
||||
<div class="col-lg-3 col-lg-offset-1 col-md-4 col-md-offset-1 col-sm-5 col-sm-offset-0 col-xs-10 col-xs-offset-1">
|
||||
<img src="/static/img/home/app-dist.png" alt="feature-app" width="100%">
|
||||
</div>
|
||||
<div class="col-lg-6 col-lg-offset-1 col-md-7 col-md-offset-0 col-sm-7 col-sm-offset-0 col-xs-12 col-xs-offset-0">
|
||||
<h2>Click and Go</h2>
|
||||
<br/>
|
||||
<p class="lead">Distributed or single node ? Never mind !
|
||||
Click it just like start an app on your smart phone, and your workspace is ready for you.</p>
|
||||
</div>
|
||||
</div>
|
||||
<hr>
|
||||
<div class="row">
|
||||
<div class="col-lg-6 col-lg-offset-1 col-md-7 col-md-offset-0 col-sm-7 col-sm-offset-0 col-xs-12 col-xs-offset-0">
|
||||
<h2>All in Web</h2>
|
||||
<br/>
|
||||
<p class="lead">All you need is a web browser. Compute in web, code in web, plot in web, anything in web !
|
||||
You can get to work anytime and anywhere by internet.</p>
|
||||
</div>
|
||||
<div class="col-lg-3 col-lg-offset-1 col-md-4 col-md-offset-1 col-sm-5 col-sm-offset-0 col-xs-10 col-xs-offset-1">
|
||||
<img src="/static/img/home/app-web.png" alt="feature-web" width="100%">
|
||||
</div>
|
||||
</div>
|
||||
<hr>
|
||||
|
||||
<footer>
|
||||
<p class="pull-right">Powered by <a href="http://docklet.unias.org" style="color:blue">Docklet</a></p>
|
||||
<p>© SEI, PKU</p>
|
||||
</footer>
|
||||
|
||||
</div>
|
||||
|
||||
<script src="http://cdn.bootcss.com/jquery/2.2.1/jquery.js"></script>
|
||||
<script src="http://cdn.bootcss.com/bootstrap/3.3.5/js/bootstrap.min.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
|
|
|
|||
|
|
@ -0,0 +1,139 @@
|
|||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
|
||||
<title>Docklet | Home</title>
|
||||
<link rel="shortcut icon" href="/static/img/favicon.ico">
|
||||
|
||||
<link href="http://cdn.bootcss.com/bootstrap/3.3.5/css/bootstrap.min.css" rel="stylesheet">
|
||||
<link href="http://cdn.bootcss.com/font-awesome/4.5.0/css/font-awesome.css" rel="stylesheet">
|
||||
|
||||
<style type="text/css">
|
||||
a.linkbtn, a.linkbtn:visited, a.linkbtn:active{
|
||||
color:white;
|
||||
padding:8px;
|
||||
}
|
||||
|
||||
a.linkbtn:hover{
|
||||
color:white;
|
||||
text-decoration:none;
|
||||
border-bottom: 1px solid white;
|
||||
|
||||
}
|
||||
.navbar-custom-top{
|
||||
background:transparent;
|
||||
position:absolute;
|
||||
z-index:1030;
|
||||
left:0px;
|
||||
right:0px;
|
||||
}
|
||||
</style>
|
||||
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<div class="navbar navbar-custom-top" role="navigation">
|
||||
<div class="container">
|
||||
<div class="row" style="font-size:16px; color:white; padding:16px">
|
||||
<div class="pull-right" >
|
||||
<a class="linkbtn" href="http://docklet.unias.org/docklet-book/userguide/_book/">Document</a>
|
||||
·
|
||||
<a class="linkbtn" href="/login/" >Sign In</a>
|
||||
</div>
|
||||
<div>
|
||||
<a class="linkbtn" href="http://docklet.unias.org"><strong>Docklet Cloud OS</strong></a>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Carousel -->
|
||||
<div id="myCarousel" class="carousel slide" data-ride="carousel" style="box-shadow:0px 2px 10px 0px black">
|
||||
<!-- Indicators -->
|
||||
<ol class="carousel-indicators">
|
||||
<li data-target="#myCarousel" data-slide-to="0" class="active"></li>
|
||||
<li data-target="#myCarousel" data-slide-to="1"></li>
|
||||
</ol>
|
||||
<div class="carousel-inner" role="listbox">
|
||||
<div class="item active">
|
||||
<img src="/static/img/home/cloud.png" alt="Cloud OS" style="min-height:600px">
|
||||
<div class="container">
|
||||
<div class="carousel-caption">
|
||||
<h1>Cloud OS</h1>
|
||||
<p class="lead">Cloud OS for your cloud Apps.</p>
|
||||
<br>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="item">
|
||||
<img src="/static/img/home/workspace.png" alt="Cloud Workspace" style="min-height:600px">
|
||||
<div class="container">
|
||||
<div class="carousel-caption">
|
||||
<h1>Cloud Workspace</h1>
|
||||
<p class="lead">Workspace in cloud, all your work in cloud.</p>
|
||||
<br>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<a class="left carousel-control" href="#myCarousel" role="button" data-slide="prev">
|
||||
<span class="glyphicon glyphicon-chevron-left" aria-hidden="true"></span>
|
||||
<span class="sr-only">Previous</span>
|
||||
</a>
|
||||
<a class="right carousel-control" href="#myCarousel" role="button" data-slide="next">
|
||||
<span class="glyphicon glyphicon-chevron-right" aria-hidden="true"></span>
|
||||
<span class="sr-only">Next</span>
|
||||
</a>
|
||||
</div><!-- /.carousel -->
|
||||
|
||||
<div class="container" style="margin-top:60px">
|
||||
<div class="row">
|
||||
<div class="col-lg-6 col-lg-offset-1 col-md-7 col-md-offset-0 col-sm-7 col-sm-offset-0 col-xs-12 col-xs-offset-0">
|
||||
<h2>Workspace=Cluster+Service+Data</h2>
|
||||
<br/>
|
||||
<p class="lead">Package service and data based on virtual cluster as virtual compute environment for your work.
|
||||
<br> This is your Workspace !</p>
|
||||
</div>
|
||||
<div class="col-lg-3 col-lg-offset-1 col-md-4 col-md-offset-1 col-sm-5 col-sm-offset-0 col-xs-10 col-xs-offset-1">
|
||||
<img src="/static/img/home/app-workspace.png" alt="feature-workspace" width="100%">
|
||||
</div>
|
||||
</div>
|
||||
<hr>
|
||||
<div class="row">
|
||||
<div class="col-lg-3 col-lg-offset-1 col-md-4 col-md-offset-1 col-sm-5 col-sm-offset-0 col-xs-10 col-xs-offset-1">
|
||||
<img src="/static/img/home/app-dist.png" alt="feature-app" width="100%">
|
||||
</div>
|
||||
<div class="col-lg-6 col-lg-offset-1 col-md-7 col-md-offset-0 col-sm-7 col-sm-offset-0 col-xs-12 col-xs-offset-0">
|
||||
<h2>Click and Go</h2>
|
||||
<br/>
|
||||
<p class="lead">Distributed or single node ? Never mind !
|
||||
Click it just like start an app on your smart phone, and your workspace is ready for you.</p>
|
||||
</div>
|
||||
</div>
|
||||
<hr>
|
||||
<div class="row">
|
||||
<div class="col-lg-6 col-lg-offset-1 col-md-7 col-md-offset-0 col-sm-7 col-sm-offset-0 col-xs-12 col-xs-offset-0">
|
||||
<h2>All in Web</h2>
|
||||
<br/>
|
||||
<p class="lead">All you need is a web browser. Compute in web, code in web, plot in web, anything in web !
|
||||
You can get to work anytime and anywhere by internet.</p>
|
||||
</div>
|
||||
<div class="col-lg-3 col-lg-offset-1 col-md-4 col-md-offset-1 col-sm-5 col-sm-offset-0 col-xs-10 col-xs-offset-1">
|
||||
<img src="/static/img/home/app-web.png" alt="feature-web" width="100%">
|
||||
</div>
|
||||
</div>
|
||||
<hr>
|
||||
|
||||
<footer>
|
||||
<p class="pull-right">Powered by <a href="http://docklet.unias.org" style="color:blue">Docklet</a></p>
|
||||
<p>© SEI, PKU</p>
|
||||
</footer>
|
||||
|
||||
</div>
|
||||
|
||||
<script src="http://cdn.bootcss.com/jquery/2.2.1/jquery.js"></script>
|
||||
<script src="http://cdn.bootcss.com/bootstrap/3.3.5/js/bootstrap.min.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -16,6 +16,50 @@
|
|||
{% endblock %}
|
||||
|
||||
{% block content %}
|
||||
<div class="row">
|
||||
<div class="col-md-12">
|
||||
<div class="box box-info">
|
||||
<div class="box-header with-border">
|
||||
<h3 class="box-title">Your Quotas</h3>
|
||||
<div class="box-tools pull-right">
|
||||
<button type="button" class="btn btn-box-tool" data-widget="collapse"><i class="fa fa-minus"></i>
|
||||
</button>
|
||||
<button type="button" class="btn btn-box-tool" data-widget="remove"><i class="fa fa-times"></i></button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="box-body table-responsive">
|
||||
|
||||
<table class="table table-bordered">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>CPU</th>
|
||||
<th>Memory</th>
|
||||
<th>Disk</th>
|
||||
<th>Vnode</th>
|
||||
<th>Image</th>
|
||||
<th>Idletime</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
{% if quotainfo['cpu'] > 1 %}
|
||||
<th>{{ quotainfo['cpu'] }} Cores</th>
|
||||
{% else %}
|
||||
<th>{{ quotainfo['cpu'] }} Core</th>
|
||||
{% endif %}
|
||||
<th>{{ quotainfo['memory'] }} MB</th>
|
||||
<th>{{ quotainfo['disk'] }} MB</th>
|
||||
<th>{{ quotainfo['vnode'] }}</th>
|
||||
<th>{{ quotainfo['image'] }}</th>
|
||||
<th>{{ quotainfo['idletime'] }} hours</th>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{% for cluster in clusters %}
|
||||
<div class="row">
|
||||
<div class="col-md-12">
|
||||
|
|
@ -106,18 +150,17 @@
|
|||
|
||||
$.post(url+"/cpu_use",{},function(data){
|
||||
var usedp = data.monitor.cpu_use.usedp;
|
||||
var quota = data.groupinfo.cpu;
|
||||
quota = quota/100000.0;
|
||||
var quota = data.monitor.cpu_use.quota.cpu;
|
||||
$("#"+index+"_cpu").html((usedp/0.01).toFixed(2)+"%");
|
||||
$("#"+index+"_cpuquota").html((quota*100).toFixed(2)+"% Cores");
|
||||
$("#"+index+"_cpuquota").html(quota+" Cores");
|
||||
},"json");
|
||||
|
||||
$.post(url+"/mem_use",{},function(data){
|
||||
var usedp = data.monitor.mem_use.usedp;
|
||||
var unit = data.monitor.mem_use.unit;
|
||||
var quota = data.groupinfo.memory;
|
||||
var quota = data.monitor.mem_use.quota.memory/1024.0;
|
||||
var val = data.monitor.mem_use.val;
|
||||
var out = "("+val+unit+"/"+quota+"MB)";
|
||||
var out = "("+val+unit+"/"+quota.toFixed(2)+"MiB)";
|
||||
$("#"+index+"_mem").html((usedp/0.01).toFixed(2)+"%<br/>"+out);
|
||||
},"json");
|
||||
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@
|
|||
<a href="/dashboard/"><i class="fa fa-dashboard"></i>Home</a>
|
||||
</li>
|
||||
<li class="active">
|
||||
<a href='/user/list/'>UserList</a>
|
||||
<strong>UserList</strong>
|
||||
</li>
|
||||
</ol>
|
||||
{% endblock %}
|
||||
|
|
|
|||
27
web/web.py
27
web/web.py
|
|
@ -20,7 +20,7 @@ from webViews.log import logger
|
|||
|
||||
from flask import Flask, request, session, render_template, redirect, send_from_directory, make_response, url_for, abort
|
||||
from webViews.dashboard import dashboardView
|
||||
from webViews.user.userlist import userlistView, useraddView, usermodifyView, groupaddView, groupdelView, userdataView, userqueryView
|
||||
from webViews.user.userlist import userlistView, useraddView, usermodifyView, userdataView, userqueryView
|
||||
from webViews.user.userinfo import userinfoView
|
||||
from webViews.user.userActivate import userActivateView
|
||||
from webViews.user.grouplist import grouplistView, groupqueryView, groupdetailView, groupmodifyView
|
||||
|
|
@ -303,9 +303,9 @@ def groupdetail():
|
|||
def groupquery():
|
||||
return groupqueryView.as_view()
|
||||
|
||||
@app.route("/group/modify/", methods=['POST'])
|
||||
@app.route("/group/modify/<groupname>/", methods=['POST'])
|
||||
@administration_required
|
||||
def groupmodify():
|
||||
def groupmodify(groupname):
|
||||
return groupmodifyView.as_view()
|
||||
|
||||
@app.route("/user/data/", methods=['GET', 'POST'])
|
||||
|
|
@ -323,6 +323,11 @@ def useradd():
|
|||
def usermodify():
|
||||
return usermodifyView.as_view()
|
||||
|
||||
@app.route("/quota/add/", methods=['POST'])
|
||||
@administration_required
|
||||
def quotaadd():
|
||||
return quotaaddView.as_view()
|
||||
|
||||
@app.route("/group/add/", methods=['POST'])
|
||||
@administration_required
|
||||
def groupadd():
|
||||
|
|
@ -394,6 +399,11 @@ def jupyter_auth(cookie_name, cookie_content):
|
|||
@app.errorhandler(401)
|
||||
def not_authorized(error):
|
||||
if "username" in session:
|
||||
if "401" in session:
|
||||
reason = session['401']
|
||||
session.pop('401', None)
|
||||
if (reason == 'Token Expired'):
|
||||
return redirect('/logout/')
|
||||
return render_template('error/401.html', mysession = session)
|
||||
else:
|
||||
return redirect('/login/')
|
||||
|
|
@ -401,7 +411,16 @@ def not_authorized(error):
|
|||
@app.errorhandler(500)
|
||||
def internal_server_error(error):
|
||||
if "username" in session:
|
||||
return render_template('error/500.html', mysession = session)
|
||||
if "500" in session and "500_title" in session:
|
||||
reason = session['500']
|
||||
title = session['500_title']
|
||||
session.pop('500', None)
|
||||
session.pop('500_title', None)
|
||||
else:
|
||||
reason = '''The server encountered something unexpected that didn't allow it to complete the request. We apologize.You can go back to
|
||||
<a href="/dashboard/">dashboard</a> or <a href="/logout">log out</a>'''
|
||||
title = 'Internal Server Error'
|
||||
return render_template('error/500.html', mysession = session, reason = reason, title = title)
|
||||
else:
|
||||
return redirect('/login/')
|
||||
if __name__ == '__main__':
|
||||
|
|
|
|||
|
|
@ -1,14 +1,40 @@
|
|||
from flask import session
|
||||
from flask import session, render_template, redirect, request
|
||||
from webViews.view import normalView
|
||||
from webViews.dockletrequest import dockletRequest
|
||||
from webViews.dashboard import *
|
||||
import time, re
|
||||
import time, re, json
|
||||
|
||||
class adminView(normalView):
|
||||
template_path = "admin.html"
|
||||
|
||||
@classmethod
|
||||
def get(self):
|
||||
groups = dockletRequest.post('/user/groupNameList/')["groups"]
|
||||
return self.render(self.template_path, groups = groups)
|
||||
result = dockletRequest.post('/user/groupList/')
|
||||
groups = result["groups"]
|
||||
quotas = result["quotas"]
|
||||
return self.render(self.template_path, groups = groups, quotas = quotas)
|
||||
|
||||
class groupaddView(normalView):
|
||||
@classmethod
|
||||
def post(self):
|
||||
dockletRequest.post('/user/groupadd', request.form)
|
||||
return redirect('/admin/')
|
||||
|
||||
class quotaaddView(normalView):
|
||||
@classmethod
|
||||
def post(self):
|
||||
dockletRequest.post('/user/quotaadd', request.form)
|
||||
return redirect('/admin/')
|
||||
|
||||
class groupdelView(normalView):
|
||||
@classmethod
|
||||
def post(self):
|
||||
data = {
|
||||
"name" : self.groupname,
|
||||
}
|
||||
dockletRequest.post('/user/groupdel', data)
|
||||
return redirect('/admin/')
|
||||
|
||||
@classmethod
|
||||
def get(self):
|
||||
return self.post()
|
||||
|
|
|
|||
|
|
@ -0,0 +1,24 @@
|
|||
import re
|
||||
from flask import abort, session
|
||||
|
||||
pattern = re.compile(r'[a-zA-Z_][a-zA-Z0-9_]*')
|
||||
error_msg = ''' Your name may cause errors, Please use names starting with a-z,A-Z or _ and contains only elements in {a-z, A-Z, _, 0-9}
|
||||
'''
|
||||
error_title = 'Input Error'
|
||||
|
||||
def checkname(str):
|
||||
try:
|
||||
match = pattern.match(str)
|
||||
if (match == None):
|
||||
session['500'] = error_msg
|
||||
session['500_title'] = error_title
|
||||
abort(500)
|
||||
if (match.group() != str):
|
||||
session['500'] = error_msg
|
||||
session['500_title'] = error_title
|
||||
abort(500)
|
||||
return True
|
||||
except:
|
||||
session['500'] = error_msg
|
||||
session['500_title'] = error_title
|
||||
abort(500)
|
||||
|
|
@ -2,6 +2,7 @@ from flask import session
|
|||
from webViews.view import normalView
|
||||
from webViews.dockletrequest import dockletRequest
|
||||
from webViews.dashboard import *
|
||||
from webViews.checkname import checkname
|
||||
import time, re
|
||||
|
||||
class addClusterView(normalView):
|
||||
|
|
@ -24,6 +25,7 @@ class createClusterView(normalView):
|
|||
def post(self):
|
||||
index1 = self.image.rindex("_")
|
||||
index2 = self.image[:index1].rindex("_")
|
||||
checkname(self.clustername)
|
||||
data = {
|
||||
"clustername": self.clustername,
|
||||
'imagename': self.image[:index2],
|
||||
|
|
@ -182,6 +184,7 @@ class detailClusterView(normalView):
|
|||
class saveImageView(normalView):
|
||||
template_path = "saveconfirm.html"
|
||||
success_path = "opsuccess.html"
|
||||
error_path = "error.html"
|
||||
|
||||
@classmethod
|
||||
def post(self):
|
||||
|
|
@ -201,7 +204,10 @@ class saveImageView(normalView):
|
|||
#res.clustername = self.clustername
|
||||
#return res.as_view()
|
||||
else:
|
||||
return self.render(self.template_path, containername = self.containername, clustername = self.clustername, image = self.imagename, user = session['username'], description = self.description)
|
||||
if result.get('reason') == "exists":
|
||||
return self.render(self.template_path, containername = self.containername, clustername = self.clustername, image = self.imagename, user = session['username'], description = self.description)
|
||||
else:
|
||||
return self.render(self.error_path, message = result.get('message'))
|
||||
else:
|
||||
self.error()
|
||||
|
||||
|
|
|
|||
|
|
@ -15,7 +15,10 @@ class dockletRequest():
|
|||
logger.info ("Docklet Request: user = %s data = %s, url = %s"%(session['username'], data, url))
|
||||
|
||||
result = requests.post(endpoint + url, data = data).json()
|
||||
if (result.get('success', None) == "false" and (result.get('reason', None) == "Unauthorized Action" or result.get('Unauthorized', None) == 'True')):
|
||||
if (result.get('success', None) == "false" and result.get('reason', None) == "Unauthorized Action"):
|
||||
abort(401)
|
||||
if (result.get('Unauthorized', None) == 'True'):
|
||||
session['401'] = 'Token Expired'
|
||||
abort(401)
|
||||
logger.info ("Docklet Response: user = %s result = %s, url = %s"%(session['username'], result, url))
|
||||
return result
|
||||
|
|
@ -24,7 +27,9 @@ class dockletRequest():
|
|||
|
||||
@classmethod
|
||||
def unauthorizedpost(self, url = '/', data = None):
|
||||
logger.info("Docklet Unauthorized Request: data = %s, url = %s" % (data, url))
|
||||
data = dict(data)
|
||||
data_log = {'user': data.get('user', 'external')}
|
||||
logger.info("Docklet Unauthorized Request: data = %s, url = %s" % (data_log, url))
|
||||
result = requests.post(endpoint + url, data = data).json()
|
||||
logger.info("Docklet Unauthorized Response: result = %s, url = %s"%(result, url))
|
||||
return result
|
||||
|
|
|
|||
|
|
@ -13,6 +13,10 @@ class statusView(normalView):
|
|||
}
|
||||
result = dockletRequest.post('/cluster/list/', data)
|
||||
clusters = result.get('clusters')
|
||||
result = dockletRequest.post('/monitor/user/quotainfo/', data)
|
||||
quotainfo = result.get('quotainfo')
|
||||
quotainfo['cpu'] = int(int(quotainfo['cpu']))
|
||||
print(quotainfo)
|
||||
if (result):
|
||||
containers = {}
|
||||
for cluster in clusters:
|
||||
|
|
@ -23,7 +27,7 @@ class statusView(normalView):
|
|||
else:
|
||||
self.error()
|
||||
containers[cluster] = message
|
||||
return self.render(self.template_path, clusters = clusters, containers = containers, user = session['username'])
|
||||
return self.render(self.template_path, clusters = clusters, quotainfo = quotainfo, containers = containers, user = session['username'])
|
||||
else:
|
||||
self.error()
|
||||
|
||||
|
|
|
|||
|
|
@ -49,21 +49,4 @@ class usermodifyView(normalView):
|
|||
return self.render('user/mailservererror.html')
|
||||
return redirect('/user/list/')
|
||||
|
||||
class groupaddView(normalView):
|
||||
@classmethod
|
||||
def post(self):
|
||||
dockletRequest.post('/user/groupadd', request.form)
|
||||
return redirect('/admin/')
|
||||
|
||||
class groupdelView(normalView):
|
||||
@classmethod
|
||||
def post(self):
|
||||
data = {
|
||||
"name" : self.groupname,
|
||||
}
|
||||
dockletRequest.post('/user/groupdel', data)
|
||||
return redirect('/admin/')
|
||||
|
||||
@classmethod
|
||||
def get(self):
|
||||
return self.post()
|
||||
|
|
|
|||
Loading…
Reference in New Issue