Commit 5bdb7f15 authored by David Haynes's avatar David Haynes 🙆
Browse files

Merge branch 'vagrant'

- :nander13: :nander13: :nander13:
parents f0edcc90 9a7c24d1
......@@ -2,6 +2,10 @@
*.swp
*.pyc
.virtualenv
.vagrant/
venv
.venv
/provisioning/playbook.retry
/go/settings/settings.py
/go/settings/secret.py
/go/static/admin
......
......@@ -6,12 +6,13 @@ Go is a drop-in URL shortening service. It aims to provide an easily
branded service for institutions that wish to widely disseminate
information without unnecessarily outsourcing branding.
## Local Enviornment Setup
I encourage you to join the #go channel in SRCT's [Slack Group](srct.slack.com) if you have any questions on setup or would like to contribute.
### Prerequisities & Required Packages
## Package Installation
<legend>
### Prerequisities
First, install python, pip, and git on your system.
* Python is the programming language used for Django, the web framework used by Go.
* 'Pip' is the python package manager.
* Git is the version control system used for SRCT projects.
Open a terminal and run the following command:
......@@ -22,163 +23,64 @@ This retrieves links to the most up-to-date and secure versions of your packages
Next, with:
`$ sudo apt-get install python python-dev python-pip git`
`$ sudo apt-get install python git`
you install python and git.
Now, we're going to clone down a copy of the Go codebase from git.gmu.edu, the SRCT code respository.
### Cloning the Go Codebase
Now, we're going to clone down a copy of the Go codebase from [git.gmu.edu](http://git.gmu.edu/srct/go), the SRCT code respository.
Configure your ssh keys by following the directions at git.gmu.edu/help/ssh/README.
Configure your ssh keys by following the directions at [git.gmu.edu/help/ssh/README](git.gmu.edu/help/ssh/README).
Now, on your computer, navigate to the directory in which you want to download the project (perhaps one called development/ or something similar), and run
`$ git clone git@git.gmu.edu:srct/go.git`
Finally, install these packages from the standard repositories:
`$ sudo apt-get install libldap2-dev mysql-server mysql-client libmysqlclient-dev python-mysqldb libsasl2-dev libjpeg-dev `
If prompted to install additional required packages, install those as well.
When prompted to set your mysql password, it's advisable to set it as the same as your normal superuser password.
### The Virtual Enviornment
Virtual environments are used to keep separate project packages from the main computer, so you can use different versions of packages across different projects and ease deployment server setup.
It's often recommended to create a special directory to store all of your virtual environments together (ie. development/virtualenv/), though they can be placed wherever is most convienent.
Run:
`$ sudo pip install virtualenv`
to install virtualenv system-wide.
Then in your virtual environment directory run:
`$ virtualenv go`
to create your virtual environment.
Activate it by typing:
`$ source go/bin/activate`
If you ever need to exit your virtual environment, simply run:
`$ deactivate`
Now, the packages you need to install for Go are in in the top level of the project's directory structure(go/).
Run:
`$ pip install -r requirements.txt`
to install all of the packages needed for the project.
### Database Configuration
Go is configured for use with a mysql database.
Load up the mysql shell by running
`$ mysql -u root -p insert_password_here`
and putting in your mysql password.
Create the database by running:
`> CREATE DATABASE go;`
You can choose a different name for your database if you desire.
Double check your database was created by running:
`> SHOW DATABASES;`
Though you can use an existing user to access this database, here's how to create a new user and give them the necessary permissions to your newly created database:
`> CREATE USER 'god'@'localhost' IDENTIFIED BY 'password';`
For local development, password strength is less important, but use a strong passphrase for deployment. You can choose a different username.
Run:
`> GRANT ALL ON go.* TO 'god'@'localhost';`
This allows your database user to create all the tables it needs on the bookshare database. (Each model in each app's models.py is a separate table, and each attribute a column, and each instance a row.)
Run:
`> GRANT ALL ON test_go.* TO 'god'@'localhost'; FLUSH PRIVILEGES;`
When running test cases, django creates a test database so your 'real' database doesn't get screwed up. This database is called 'test_' + whatever your normal database is named. Note that for permissions it doesn't matter that this database hasn't yet been created.
The .* is to grant access all tables in the database, and 'flush privileges' reloads privileges to ensure that your user is ready to go.
Exit the mysql shell by typing:
`> exit`
### Additional Setup
Now, to configure your newly created database with the project settings, and set up your project's cryptographic key, copy the secret.py.template in settings/ to secret.py. Follow the comment instructions provided in each file to set your secret key and database info.
Also copy settings.py.template to settings.py. You will need to set DEBUG mode to True in order to view more details when things go awry.
Change directory into go/go/ and run:
`$ python manage.py makemigrations`
to create the tables and rows and columns for your database. This command generates sql code based on your database models.
Then run:
`$ python manage.py migrate`
### Required Packages
Finally, install these packages from the standard repositories:
- VirtualBox
to execute that sql code and set up your database. Migrations also track how you've changed your models over the life of your database, which allows you to make changes to your tables without screwing up existing information.
`$ sudo apt-get install virtualbox`
- Vagrant
Finally, run:
`$ sudo apt-get install vagrant`
- Ansible
`$ python manage.py createsuperuser`
`$ sudo easy_install pip && sudo pip install ansible`
to create an admin account, using the same username and email as you'll access through CAS. This means your 'full' email address, for instance gmason@masonlive.gmu.edu. Your password will be handled through CAS, so you can just use 'password' here.
## Developing with Vagrant
<legend></legend>
Vagrant allows for the virtualization of your development enviornment and automates the setup process for Go.
(If you accidentally skip this step, you can run python manage.py shell and edit your user from there. Select your user, and set .is_staff and .is_superuser to True, then save.)
### Vagrant Setup
Navigate to go/ and run:
`$ vagrant up`
Finally, within 'go/go' run:
This will setup a vm to run Go on your computer and will setup a database, install packages, etc. The first time you run `vagrant up` it may take a few minutes to setup, specifically when installing Go packages. Don't worry as progressive times it will speed up.
`$ python manage.py runserver`
And that's it! Navigate to [localhost](127.0.0.1:8000) in your web browser to view the website.
to start the site. Open your preferred web browser and navigate to 127.0.0.1:8000/ to see the site!
### Additional Notes
If you make any changes to _models.py_ you will need to re-provision the vm:
In order to approve yourself to be an 'approved user' you must navigate to 127.0.0.1:8000/admin and log in. Once in the admin page go to "registered users", and create a new registered user in the top right. Be sure to use the same username and Full Name as your main account and select "approved" in the bottom row.
`$ vagrant provision`
## To Do
* qr codes on links view-- need to save the pictures somewhere, render
inline as well as in different formats and sizes for download, and be
deleted along with the links
* Update the user authentication system (ie. port it to CAS to play nicely
with GMU)
* Update the user registration system. Make it more intuitive to first time
users, also update the connection between registered users and actual
user auth accounts on the database.
* Set up Piwik to work with Go.
* Update the documentation on Go to include a setup guide
* Update the interface to bootswatch, perhaps? Maybe the same stylesheet as
is used on SRCTWeb. (ie. complete HTML overhaul)
* Remove all Mason-specific branding.
Please note that this will refresh the database.
## Notes for Production
It is also good practice to shutdown your vm when you are done:
### settings.py
`$ vagrant halt`
The settings file should already be configured acceptably. You may need to
add a different authentication backend or database engine.
and to restart with:
### nginx / Apache
`$ vagrant up`
You must configure an outside web server to properly host the static file
required to run this Django app.
## On Deployemnt
<legend>
### Deploying with Vagrant
TODO
### Cron
......
# -*- mode: ruby -*-
# vi: set ft=ruby :
# Import securerandom for secret key generation
require 'securerandom'
# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.
Vagrant.configure(2) do |config|
# The most common configuration options are documented and commented below.
# For a complete reference, please see the online documentation at
# https://docs.vagrantup.com.
# Every Vagrant development environment requires a box. You can search for
# boxes at https://atlas.hashicorp.com/search.
config.vm.box = "ubuntu/trusty64"
# Disable automatic box update checking. If you disable this, then
# boxes will only be checked for updates when the user runs
# `vagrant box outdated`. This is not recommended.
# config.vm.box_check_update = false
# Create a forwarded port mapping which allows access to a specific port
# within the machine from a port on the host machine. In the example below,
# accessing "localhost:8080" will access port 80 on the guest machine.
config.vm.network "forwarded_port", guest: 8000, host: 8000
# Create a private network, which allows host-only access to the machine
# using a specific IP.
# config.vm.network "private_network", ip: "192.168.33.10"
# Create a public network, which generally matched to bridged network.
# Bridged networks make the machine appear as another physical device on
# your network.
# config.vm.network "public_network"
# Share an additional folder to the guest VM. The first argument is
# the path on the host to the actual folder. The second argument is
# the path on the guest to mount the folder. And the optional third
# argument is a set of non-required options.
# config.vm.synced_folder "../data", "/vagrant_data"
# Provider-specific configuration so you can fine-tune various
# backing providers for Vagrant. These expose provider-specific options.
# Example for VirtualBox:
#
# config.vm.provider "virtualbox" do |vb|
# # Display the VirtualBox GUI when booting the machine
# vb.gui = true
#
# # Customize the amount of memory on the VM:
# vb.memory = "1024"
# end
#
# View the documentation for the provider you are using for more
# information on available options.
# Define a Vagrant Push strategy for pushing to Atlas. Other push strategies
# such as FTP and Heroku are also available. See the documentation at
# https://docs.vagrantup.com/v2/push/atlas.html for more information.
# config.push.define "atlas" do |push|
# push.app = "YOUR_ATLAS_USERNAME/YOUR_APPLICATION_NAME"
# end
# Enable provisioning with a shell script. Additional provisioners such as
# Puppet, Chef, Ansible, Salt, and Docker are also available. Please see the
# documentation for more information about their specific syntax and use.
# config.vm.provision "shell", inline: <<-SHELL
# sudo apt-get update
# sudo apt-get install -y apache2
# SHELL
config.vm.provision "ansible", run: "always" do |ansible|
ansible.playbook = "provisioning/playbook.yml"
ansible.sudo = true
ansible.extra_vars = {
nginx: {
port: 8000,
https: false
},
mysql: {
root_user: "root",
root_pass: "",
user: "go",
pass: "go",
db: "go"
},
django: {
secret_key: SecureRandom.base64,
host: "*",
debug: "True",
email_domain: "@masonlive.gmu.edu",
requirements_path: "/vagrant/requirements.txt",
venv_path: "/vagrant/venv",
cas_url: "https://nanderson.me/cas/",
app_path: "/vagrant/go",
settings_path: "/vagrant/go/settings",
superuser: "dhaynes3"
}
}
end
=begin
# Picks up from any failed runs
# Run this with: "vagrant provision --provision-with resume"
config.vm.provision "resume", type: "ansible" do |resume|
resume.playbook = "provisioning/playbook.yml"
resume.limit = "@provisioning/playbook.retry"
resume.sudo = true
resume.extra_vars = {
nginx: {
port: 8000,
https: false
},
mysql: {
root_user: "root",
root_pass: "",
user: "go",
pass: "go",
db: "go"
},
django: {
secret_key: SecureRandom.base64,
host: "*",
debug: "True",
email_domain: "@masonlive.gmu.edu",
requirements_path: "/vagrant/requirements.txt",
venv_path: "/vagrant/venv",
cas_url: "https://nanderson.me/cas",
app_path: "/vagrant/go",
settings_path: "/vagrant/go/settings",
superuser: "dhaynes3"
}
}
end
=end
end
---
# Ansible Playbook for Go
#
# Installs a production-like envrionment with:
# - MySQL
# - nginx
# - uwsgi + Emperor
# - Django
- hosts: all
tasks:
- name: install go packages
apt: name={{ item }} state=latest
with_items:
- python
- python-dev
- python-virtualenv
- python-pip
- git
- libldap2-dev
- mysql-server
- mysql-client
- libmysqlclient-dev
- python-mysqldb
- libsasl2-dev
- libjpeg-dev
- name: create mysql user
mysql_user: >
name={{ mysql['user'] }}
password={{ mysql['pass'] }}
login_password={{ mysql['root_pass'] }}
login_user={{ mysql['root_user'] }}
state=present
host=localhost
priv={{ mysql['db'] }}.*:ALL
- name: create mysql database
mysql_db: >
name={{ mysql['db'] }}
state=present
login_user={{ mysql['root_user'] }}
login_password={{ mysql['root_pass'] }}
- name: install site packages to virtual env
pip: >
requirements={{ django['requirements_path'] }}
virtualenv={{ django['venv_path'] }}
virtualenv_python=python2.7
- name: install django settings.py
template: >
src=templates/settings.py.j2
dest={{ django['settings_path'] }}/settings.py
backup=yes
- name: install django secret.py
template: >
src=templates/secret.py.j2
dest={{ django['settings_path'] }}/secret.py
- name: setup django database
django_manage: >
command={{ item }}
app_path={{ django['app_path'] }}
virtualenv={{ django['venv_path'] }}
with_items:
- flush
- makemigrations
- makemigrations go
- migrate
- name: create python superuser (defaults to dhaynes3)
django_manage: >
command="createsuperuser --noinput --username={{ django['superuser'] }} --email={{ django['superuser'] }}@masonlive.gmu.edu"
app_path={{ django['app_path'] }}
virtualenv={{ django['venv_path'] }}
- name: killing old screen sessions
shell: screen -ls | grep -q django && screen -X -S django quit
ignore_errors: true
- name: start django runserver (access via localhost:8000)
command: screen -dmS django bash -c "echo Starting on port {{ nginx['port'] }}; cd /vagrant/go; source ../venv/bin/activate; python manage.py runserver 0.0.0.0:8000;"
# Create a new file 'secret.py' and copy these contents into that file
# Please be sure to keep these variables secret in production
# You can generate a secret key from the following link: http://www.miniwebtool.com/django-secret-key-generator/
SECRET_KEY = "{{ django['secret_key'] }}"
# Use the values from the database configuration
DB_NAME = "{{ mysql['db'] }}"
DB_USER = "{{ mysql['user'] }}"
# Remember to use a strong password in production
DB_PASSWORD = "{{ mysql['pass'] }}"
# Often left blank
DB_HOST = ""
# Set piwik server site id (piwik can track multiple websites)
PIWIK_SITE_ID = ""
# Point to the piwik url
PIWIK_URL = ""
# Email configuration, if necessary
EMAIL_HOST = ""
EMAIL_PORT = ""
EMAIL_HOST_USER = ""
EMAIL_HOST_PASSWORD = ""
# Create a new file 'settings.py' and copy these contents into that file
import secret
import os
AUTH_MODE = "CAS"
BASE_DIR = os.path.dirname(os.path.dirname(__file__))
# DEBUG mode is used to view more details when errors occur
# Do not have set True in production
DEBUG = {{ django['debug'] }}
ADMINS = ()
MANAGERS = ADMINS
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': secret.DB_NAME,
'USER': secret.DB_USER,
'PASSWORD': secret.DB_PASSWORD,
'HOST': secret.DB_HOST,
'PORT': '',
}
}
# The domains this application will be deployed on
# e.g. Which domains this app should listen to requests from.
ALLOWED_HOSTS = ['{{ django['host'] }}']
# Peoplefinder API
PF_URL = "http://api.srct.gmu.edu/pf/v1/"
TIME_ZONE = 'America/New_York'
LANGUAGE_CODE = 'en-us'
SITE_ID = 1
USE_I18N = True
USE_L10N = True
USE_TZ = True
MEDIA_URL = '/media/'
MEDIA_ROOT = ''
MEDIAFILES_DIRS = (
os.path.join(BASE_DIR, 'media/'),
)
STATIC_URL = '/static/'
STATIC_ROOT = ''
STATICFILES_DIRS = (
os.path.join(BASE_DIR, 'static/'),
)
STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
)
SECRET_KEY = secret.SECRET_KEY
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [
os.path.join(BASE_DIR, 'templates')
],
'OPTIONS': {
'context_processors': [
'django.contrib.auth.context_processors.auth',
'django.template.context_processors.request'
],
'loaders': [
'django.template.loaders.app_directories.Loader'
],
'debug': DEBUG
}
}
]
MIDDLEWARE_CLASSES = (
'django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
)
ROOT_URLCONF = 'settings.urls'
WSGI_APPLICATION = 'settings.wsgi.application'
INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'go',
'piwik',
'django.contrib.admin',
'qrcode',
'crispy_forms',
)
SESSION_SERIALIZER = 'django.contrib.sessions.serializers.JSONSerializer'
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse'
}
},
'handlers': {
'mail_admins': {
'level': 'ERROR',
'filters': ['require_debug_false'],
'class': 'django.utils.log.AdminEmailHandler'
}
},
'loggers': {
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
}
}
LOGIN_URL = '/login'
LOGOUT_URL = '/logout'
LOGIN_REDIRECT_URL = '/'
AUTHENTICATION_BACKENDS = (
'django.contrib.auth.backends.ModelBackend',
)
if AUTH_MODE.lower() == 'ldap':
import ldap
AUTHENTICATION_BACKENDS += (
'django_auth_ldap.backend.LDAPBackend',
)
AUTH_LDAP_SERVER_URI = "ldaps://directory.gmu.edu:636" # server url
AUTH_LDAP_BIND_DN = "ou=people,o=gmu.edu" # bind DN
AUTH_LDAP_BIND_AS_AUTHENTICATING_USER = True # use the user
AUTH_LDAP_USER_DN_TEMPLATE = "uid=%(user)s,ou=people,o=gmu.edu"
AUTH_LDAP_GLOBAL_OPTIONS = { # ignore UAC cert.
ldap.OPT_X_TLS: ldap.OPT_X_TLS_DEMAND,
ldap.OPT_X_TLS_REQUIRE_CERT: ldap.OPT_X_TLS_NEVER,
}
AUTH_LDAP_USER_ATTR_MAP = {
"first_name": "givenName",
"last_name": "sn",
"email": "mail"
}
AUTH_LDAP_ALWAYS_UPDATE_USER = True
elif AUTH_MODE.lower() == 'cas':
CAS_SERVER_URL = "{{ django['cas_url'] }}"
CAS_LOGOUT_COMPLETELY = True
CAS_PROVIDE_URL_TO_LOGOUT = True
AUTHENTICATION_BACKENDS += (
'cas.backends.CASBackend',
)
CAS_RESPONSE_CALLBACKS = (
'go.cas_callbacks.create_user',
)