Release 0.7.0

.gitignore

@@ -11,3 +11,4 @@
 /rpm/vmango.spec
 /rpm/*.tar.gz
 /deb/*.tar.gz
+/.vscode/

README.md

@@ -20,11 +20,11 @@ Current features:
 * IP address management
 * Simple API
 * Custom userdata for cloud-init
+* Bridged network
 
 Hypervisor server requirements:
 
 * Libvirt 0.10+ (centos6+, ubuntu14.04+, debian8+)
-* Routed network with libvirt managed dhcp server. Bridged networks not supported due to impossibility to determine machine ip address.
 
 Web interface server requirements:
 * Libvirt 1.2.0+ (Ubuntu 14.04+, debian8+, centos7+)
@@ -45,14 +45,14 @@ Define libvirt network
     virsh net-autostart vmango
 
 Define libvirt images storage:
-    
+
     sudo mkdir -p /var/lib/libvirt/images/vmango-images
     virsh pool-define storage-pool-images.xml
     virsh pool-start vmango-images
     virsh pool-autostart vmango-images
 
 Install dhcp lease monitor hook (symlink doesn't work due to Apparmor restrictions):
-    
+
     sudo cp qemu-hook-lease-monitor.py /etc/libvirt/hooks/qemu
 
 Download vm images (file names matter!)
@@ -66,7 +66,7 @@ Download vm images (file names matter!)
 
 If your processor doesn't support hardware acceleration, change type from "kvm" to "qemu" in the first line of vm.xml.in (or you will get an error during first machine creation):
 
-    <domain type='qemu'> 
+    <domain type='qemu'>
 
 ### Dependencies for Ubuntu 14.04+
 

debian/changelog

@@ -1,3 +1,10 @@
+vmango (0.7.0) vmango; urgency=medium
+
+  * Add support for scripted network configuration
+  * Add request method to access log
+
+ -- Matvey Kruglov <kubuzzzz@gmail.com>  Mon, 12 Feb 2018 23:07:14 +0300
+
 vmango (0.6.0) vmango; urgency=medium
 
   * Preserve machine plan, creator and source image for virtual machines

docs/Makefile

@@ -1,11 +1,12 @@
 DEPLOY_HOST=no_host_set
 DEPLOY_PATH=/srv/www/vmango.org/docs/
+HUGO = hugo-0.21
 
 dev:
-	hugo server -D -E
+	$(HUGO) server -D -E
 
 deploy: clean
-	hugo
+	$(HUGO)
 	rsync -v --delete-after -a public/ $(DEPLOY_HOST):$(DEPLOY_PATH)
 
 clean:

docs/content/configuration/hypervisor.md

@@ -1,6 +1,6 @@
 +++
 weight = 10
-title = "Livirt Hypervisor"
+title = "Libvirt Hypervisor"
 date = "2017-02-09T23:36:17+03:00"
 toc = true
 +++
@@ -108,6 +108,53 @@ Nat network XML example:
 
 But, if you plan to use any VPN software (e.g. [OpenVPN](https://openvpn.net/)) to access machines on this server, don't use `<forward mode="nat"/>`, instead use `<forward mode="route"/>` with custom NAT iptables rule added on your server manually. In case of "nat" mode, libvirt adds iptables restrictions that may prevent you from access private network over VPN.
 
+### Scripted network
+
+If you want to use your own network configuration like bridge interfaces to expose virtual machines, you should use this type of network. Set hypervisor.network_script to path to executable and hypervisor.network to bridge name, e.g:
+
+```
+    hypervisor "REMOTE" {
+      url = "qemu+ssh://virt@192.168.84.19/system?socket=/var/run/libvirt/libvirt-sock"
+      image_storage_pool = "vmango-images"
+      root_storage_pool = "default"
+      network = "br0"
+      network_script = "./network_scripts/network_mikrotik.py"
+      vm_template = "vm.xml.in"
+      volume_template = "volume.xml.in"
+    }
+```
+
+And add bridge name to machine xml configuration template (vm.xml.in):
+
+```xml
+<domain type='kvm'>
+  ...
+  <devices>
+    ...
+    <interface type='bridge'>
+      <source bridge='{{ .Network }}'/>
+      <model type='virtio'/>
+    </interface>
+    ...
+  </devices>
+  ...
+</domain>
+```
+
+Vmango will call network_script on machine creation, machine deletion and to fetch current machine ip address on each API call or machine view. So, script should implement three actions:
+
+  - lookup-ip
+  - assign-ip
+  - release-ip
+
+Machine parameters can be fetched via environment variables with prefix `VMANGO_`, like `VMANGO_MACHINE_HWADDR`.
+
+To better understand what such scripts should be look like, check examples:
+
+  - [Testing stub](https://github.com/subuk/vmango/blob/master/fixtures/stub_network_script.py)
+  - [Mikrotik routers](https://github.com/subuk/vmango/blob/master/network_scripts/network_mikrotik.py)
+
+
 ## Storage
 
 You need two pools on each server. One for images and one for machine drives. Image pool should always be a directory, for machine drives you can use directory or LVM volume group.
@@ -191,7 +238,7 @@ If there any errors during image name parsing, it will be skipped with warning i
 
 Two templates are used to tell Vmango how to create new machines on hypervisor. The first one is a domain xml template ([hypervisor.vm_template]({{% ref "vmango.conf.md#hypervisor" %}}) option), the second is a machine's root volume template ([hypervisor.volume_template]({{% ref "vmango.conf.md#hypervisor" %}})). By customizing this templates you can use any storage configuration and hypervisor driver supported by libvirt, but currently only QEMU/KVM machines with LVM or qcow2 storages are tested. Feel free to experiment and send pull requests for other configurations.
 
-The exact template content depends on your system configuration, but examples are shipped with deb/rpm packages ([vm.dist.xml.in](https://github.com/subuk/vmango/blob/master/vm.dist.xml.in) and [volume.dist.xml.in](https://github.com/subuk/vmango/blob/master/volume.dist.xml.in)). You may also look at [test fixtures](https://github.com/subuk/vmango/tree/master/fixtures/libvirt). 
+The exact template content depends on your system configuration, but examples are shipped with deb/rpm packages ([vm.dist.xml.in](https://github.com/subuk/vmango/blob/master/vm.dist.xml.in) and [volume.dist.xml.in](https://github.com/subuk/vmango/blob/master/volume.dist.xml.in)). You may also look at [test fixtures](https://github.com/subuk/vmango/tree/master/fixtures/libvirt).
 
 When customizing templates, remember that filename of root drive must ends with `_disk` suffix.
 

docs/content/configuration/vmango.conf.md

@@ -13,7 +13,7 @@ All vm configuration templates are golang text templates: https://golang.org/pkg
 
 **listen** - Web server listen address
 
-**session_secret** - Secret key for session cookie encryption. 
+**session_secret** - Secret key for session cookie encryption.
 
 **static_cache** - Static files cache duration, e.g: "1d", "10m", "60s". Used mainly for development.
 
@@ -35,33 +35,35 @@ Libvirt socket path can be changed via ?socket=/path/to/libvirt-sock url option.
 
 **hypervisor** - Hypervisor definition, may be specified multiple times.
 
-**hypervisor.url** - Libvirt connection URL. 
+**hypervisor.url** - Libvirt connection URL.
 
 **hypervisor.image_storage_pool** - Libvirt storage pool name for VM images.
 
 **hypervisor.root_storage_pool** - Libvirt storage pool name for root disks.
 
 **hypervisor.ignored_vms** - List of ignored virtual machines names.
 
-**hypervisor.network** - Libvirt network name.
+**hypervisor.network** - Libvirt network name or bridge name if network_script specified.
+
+**hypervisor.network_script** - Path to executable file to integrate vmango with network not managed by libvirt (see [network]({{% ref "hypervisor.md#scripted-network" %}}) chapter).
 
 **hypervisor.vm_template** - Path to go template file (relative to vmango.conf) with libvirt domain XML. Used to create a new machine.
 
 Execution context:
 
-* Machine    [VirtualMachine](https://github.com/subuk/vmango/blob/master/src/vmango/models/vm.go#L56) 
-* Image  - [Image](https://github.com/subuk/vmango/blob/master/src/vmango/models/image.go#L18)
-* Plan       [Plan](https://github.com/subuk/vmango/blob/master/src/vmango/models/plan.go#L3)
+* Machine    [VirtualMachine](https://github.com/subuk/vmango/blob/master/src/vmango/domain/vm.go#L57)
+* Image  - [Image](https://github.com/subuk/vmango/blob/master/src/vmango/domain/image.go#L16)
+* Plan       [Plan](https://github.com/subuk/vmango/blob/master/src/vmango/domain/plan.go#L3)
 * VolumePath string
 * Network    string
 
 **hypervisor.volume_template** - Path to go template file (relative to vmango.conf) with libvirt volume XML. It is used to create a new root volume for a new machine.
 
 Execution context:
 
-* Machine    [VirtualMachine](https://github.com/subuk/vmango/blob/master/src/vmango/models/vm.go#L56) 
-* Image  - [Image](https://github.com/subuk/vmango/blob/master/src/vmango/models/image.go#L18)
-* Plan       [Plan](https://github.com/subuk/vmango/blob/master/src/vmango/models/plan.go#L3)
+* Machine    [VirtualMachine](https://github.com/subuk/vmango/blob/master/src/vmango/domain/vm.go#L57)
+* Image  - [Image](https://github.com/subuk/vmango/blob/master/src/vmango/domain/image.go#L16)
+* Plan       [Plan](https://github.com/subuk/vmango/blob/master/src/vmango/domain/plan.go#L3)
 
 Example:
 

docs/content/installation/packages.md

@@ -15,20 +15,20 @@ The simplest way to install.
 For all distributions, configuration files located in /etc/vmango, logs managed by init system (systemd or upstart).
 
 
-#### Ubuntu 14.04/16.04
+#### Ubuntu 14.04 / 16.04
 
+    sudo apt-get install apt-transport-https gnupg lsb-release
     echo deb https://dl.vmango.org/ubuntu $(lsb_release -c -s) main |sudo tee /etc/apt/sources.list.d/vmango.list
     wget -O- https://dl.vmango.org/repo.key | sudo apt-key add -
-    sudo apt-get install apt-transport-https
     sudo apt-get update
     sudo apt-get install vmango
 
 
-#### Debian 8
+#### Debian 8 / 9
 
-    echo deb https://dl.vmango.org/debian jessie main |sudo tee /etc/apt/sources.list.d/vmango.list
+    sudo apt-get install apt-transport-https gnupg lsb-release
+    echo deb https://dl.vmango.org/debian $(lsb_release -c -s) main |sudo tee /etc/apt/sources.list.d/vmango.list
     wget -O- https://dl.vmango.org/repo.key | sudo apt-key add -
-    sudo apt-get install apt-transport-https
     sudo apt-get update
     sudo apt-get install vmango
 

docs/layouts/index.html

@@ -50,7 +50,7 @@ <h1>Vmango</h1>
           It uses openstack compatible image format, is simple to install and stores all information
           about machines directly inside the hypervisor. If you have a handful of servers with
           virtual machines and don't want to install complex systems like <a href="https://www.openstack.org">Openstack</a> or <a href="https://www.ovirt.org">oVirt</a>,
-          you definitly should try it.
+          you definitely should try it.
         </p>
         <br>
         <p>

fixtures/stub_network_script.py

@@ -0,0 +1,45 @@
+#!/usr/bin/env python
+from __future__ import print_function
+import os
+import sys
+
+
+def lookup():
+    assert "VMANGO_MACHINE_HWADDR" in os.environ
+    assert "VMANGO_MACHINE_NAME" in os.environ
+    assert "VMANGO_MACHINE_PLAN" in os.environ
+    assert "VMANGO_MACHINE_ID" in os.environ
+    print("44.43.42.41")
+
+
+def assign():
+    assert "VMANGO_MACHINE_HWADDR" in os.environ
+    assert "VMANGO_MACHINE_NAME" in os.environ
+    assert "VMANGO_MACHINE_PLAN" in os.environ
+    assert "VMANGO_MACHINE_ID" in os.environ
+    print("44.43.42.41")
+
+
+def release():
+    assert "VMANGO_MACHINE_HWADDR" in os.environ
+    assert "VMANGO_MACHINE_IP" in os.environ
+    assert "VMANGO_MACHINE_NAME" in os.environ
+    assert "VMANGO_MACHINE_PLAN" in os.environ
+    assert "VMANGO_MACHINE_ID" in os.environ
+
+
+def error():
+    sys.stderr.write("Unknown action requested\n")
+    return 1
+
+def main():
+    handlers = {
+        "lookup-ip": lookup,
+        "assign-ip": assign,
+        "release-ip": release,
+    }
+    return handlers.get(sys.argv[1], error)()
+
+
+if __name__ == '__main__':
+    sys.exit(main() or 0)