Thursday, September 28, 2017

First Impressions of Anka Run/Flow

I recently found out about Anka Run/Flow, which uses macOS' native Hypervisor.Framework and includes Veertu's proprietary tools for working with virtual environments. Simplest way I'd describe it would be Docker for macOS guests. Haven't seen much media coverage of it, so here's my quick mini-review/setup of it.

(Most of these commands can also be found at: https://ankadoc.bitbucket.io/)

First, grab the Anka installer here: https://veertu.com/download-anka-run/ Install it, and get a 30-day trial key from: https://veertu.com/anka-run-trial/

mbp~ dave$ sudo anka license activate 2093-7122-6646-6940
Password:
License activated
mbp:~ dave$ anka license show
+---------------------+---------------------+
| license_type        | com.veertu.anka.run |
+---------------------+---------------------+
| status              | valid               |
+---------------------+---------------------+
| expires             | 28-oct-2017         |
+---------------------+---------------------+
| max_number_of_cores | 32                  |
+---------------------+---------------------+

You'll need to download a macOS installer to use as the VM base. This method will automatically install/include the needed paravirtual drivers, unlike the AutoDMG and ISO method (which requires manual intervention after).



When creating the first VM on a High Sierra host, you'll get a few warnings about blocked extensions. Go to "Security & Privacy" as usual and unblock them as needed.

mbp:~ dave$ anka create --ram-size 3G --cpu-count 4 --disk-size 20G --app /Applications/Install\ macOS\ High\ Sierra.app highsierra
Installing macOS 10.13...
Copying addons... 100.%
Converting to ANKA format...
20480+0 records in
Preformatting hard drive
vm created successfully with uuid: 4c4c7b33-a452-11e7-821e-7a008970c401

For High Sierra guests, a few more commands are needed on the first boot.

mbp:~ dave$ anka start highsierra
+-----------------------+--------------------------------------+
| uuid                  | 4c4c7b33-a452-11e7-821e-7a008970c401 |
+-----------------------+--------------------------------------+
| ram                   | 3G                                   |
+-----------------------+--------------------------------------+
| name                  | highsierra                           |
+-----------------------+--------------------------------------+
| cpu_cores             | 4                                    |
+-----------------------+--------------------------------------+
| hard_drive            | 20 GB (9493893120 bytes on disk)     |
+-----------------------+--------------------------------------+
| status                | running                              |
+-----------------------+--------------------------------------+
| vnc_connection_string | vnc://:[email protected]:5900     |
+-----------------------+--------------------------------------+
| view_vm_display       | anka view highsierra                 |
+-----------------------+--------------------------------------+
mbp:~ dave$ anka run highsierra sudo kextcache -system-prelinked-kernel
KernelCache ID: 7B35CF37ACB0190023E5D78FEAC2684C
mbp:~ dave$ anka stop highsierra
VM highsierra is shutting down

Now, here's my one negative comment: VNC is listening on 0.0.0.0 by default.. There's two ways to fix that:

mbp:~ dave$ # Option 1: Bind VNC to 127.0.0.1.
mbp:~ dave$ \
UUID=$(anka --machine-readable list | python -c 'import json,sys;obj=json.load(sys.stdin); print obj["body"][0]["uuid"];') && sed -i '' 's/vnc_ip: 0.0.0.0/vnc_ip: 127.0.0.1/' "$(anka config -l vm_lib_dir)"/${UUID}/${UUID}.yaml && UUID=""

mbp:~ dave$ anka modify highsierra set vnc 0 # Option 2, disable VNC completely
the following properties were set successfully:
vnc set to 0

You can now start up the VM and connect to it.


The syntax is similar to Docker and is self-explanatory.

mbp:~ dave$ anka run highsierra uname -a
Darwin Mac-mini.local 17.0.0 Darwin Kernel Version 17.0.0: Thu Aug 24 21:48:19 PDT 2017; root:xnu-4570.1.46~2/RELEASE_X86_64 x86_64

Positives:
  • Extremely simple and intuitive command line interface (takes <15 minutes from install to a fully working VM).
  • Guest VM install process is completely automated.
  • Supports real USB devices (will test in a future blog post).
  • Feels much faster than VMware Fusion (will test and confirm in a future blog post).
  • Optional Anka Registry for maintaining macOS images (will test in a future blog post).
Negatives:

Wednesday, September 27, 2017

VMware Fusion 10 - Introduction - Rest API

VMware released Fusion 10 earlier today, and introduced a new REST API. Unfortunately, they didn't include much of an introduction on how to use it..

I've copied the API docs onto GitHub: https://manouchehri.github.io/VMware-Fusion-REST-API/

mbp:~ dave$ /Applications/VMware\ Fusion.app/Contents/Public/vmrest -h
VMware Fusion REST API
Copyright (C) 2015-2017 VMware Inc.
All Rights Reserved

vmrest 1.0.0 build 6665085
Usage of /Applications/VMware Fusion.app/Contents/Public/vmrest:
  -c, --cert-path <cert-path>
    REST API Server certificate path
  -C, --config
    Configure credential
  -d, --debug
    Enable debug logging
  -h, --help
    Print usage
  -i, --ip <ip>
    REST API Server IP binding (default 127.0.0.1)
  -k, --key-path <key-path>
    REST API Server private key path
  -p, --port <port>
    REST API Server port (default 8697)
  -v, --version
    Print version information

To enable the VMware Fusion REST API, you first need to set a username and password.

mbp:~ dave$ /Applications/VMware\ Fusion.app/Contents/Public/vmrest -d -C
VMware Fusion REST API
Copyright (C) 2015-2017 VMware Inc.
All Rights Reserved

vmrest 1.0.0 build 6665085
Username:dave
New password:
Retype new password:
Processing...
Credential updated successfully

Now launching the service should work.

mbp:~ dave$ /Applications/VMware\ Fusion.app/Contents/Public/vmrest -d
VMware Fusion REST API
Copyright (C) 2015-2017 VMware Inc.
All Rights Reserved

vmrest 1.0.0 build 6665085
-
Using the Fusion UI while API calls are in progress is not recommended and may yield unexpected or unintended results.
-
Serving HTTP on 127.0.0.1:8697

And finally, you can find the actual documentation at http://127.0.0.1:8697. Logging in with your new username and password should work too.











Saturday, September 16, 2017

strongSwan with PureVPN (IKEv2/IPsec)

For those who prefer IPSec over OpenVPN, here's a quick guide on setting strongSwan up with PureVPN.

Note: While PureVPN only has 3DES enabled for IPSec tunnels, we are mitigating Sweet32 (birthday attack) by rekeying every <32GB.

Installing strongSwan

sudo apt-get -y install strongswan strongswan-plugin-eap-mschapv2

Installing Certificates

curl "https://support.comodo.com/index.php?/Knowledgebase/Article/GetAttachment/970/821027" | openssl x509 | sudo tee /etc/ipsec.d/cacerts/comodo-rsa-domain-validation-secure-server-ca.pem | openssl x509 -text
curl "https://support.comodo.com/index.php?/Knowledgebase/Article/GetAttachment/969/821026" | openssl x509 | sudo tee /etc/ipsec.d/cacerts/comodo-rsa-certification-authority.pem | openssl x509 -text

/etc/ipsec.secrets

# ADD USERNAME AND PASSWORD ON LINE BELOW
purevpn0sXXXXXXX : EAP "PUT_YOUR_PASSWORD_HERE"

# https://wiki.strongswan.org/projects/strongswan/wiki/EapSecret

/etc/ipsec.conf

config setup
uniqueids=yes
charondebug="ike 2, knl 1, cfg 1, net 1, esp 1, dmn 1, mgr 1"

conn %default
fragmentation=yes
rekey=yes
keyingtries=%forever
keyexchange=ikev2
compress=no
dpddelay=30s
dpdtimeout=90s
dpdaction=restart
closeaction=restart
# leftfirewall=yes

conn purevpn
# ADD USERNAME ON LINE BELOW
eap_identity=purevpn0sXXXXXXX

# ADD SERVER ON LINE BELOW
right=cato.pointtoserver.com
# https://support.purevpn.com/vpn-servers

# ADD RANDOM UNIQUE USERNAME ID ON LINE BELOW
leftid=@tXzjX7rLpXjT

left=%any
leftsourceip=%config4
leftsubnet=%dynamic
leftauth=eap-mschapv2
leftsendcert=never

rightid="OU=Domain Control Validated, OU=PositiveSSL Multi-Domain, CN=PointtoServer.com"
rightauth=pubkey
rightsendcert=never
rightsubnet=0.0.0.0/0

# Ciphers for Windows Server 2008
ike=3des-sha1-modp1024!
esp=3des-sha1!
# Source: https://technet.microsoft.com/en-us/library/dd125380(v=ws.10).aspx

# Do not send more than 32GB without rekeying due to SWEET32.
lifebytes=32000000000
# Sources:
# https://live.paloaltonetworks.com/t5/Threat-Vulnerability-Articles/Information-on-Sweet32-for-Palo-Alto-Networks-Customers/ta-p/128526
# https://sweet32.info/

# 8GB
marginbytes=80000000000
# This should be raised for >10Gbps connections

# 2^32
lifepackets=4294967296
# Note: This situation shouldn't be possible if lifebytes is correctly set.

# 2^28
marginpackets=268435456
# This should be raised for >10Gbps connections

# Don't waste bandwidth rekeying when it's not needed
lifetime=0
ikelifetime=0
reauth=no

# We cannot use auto=route with leftsourceip.
auto=start


Friday, September 8, 2017

Fast IPSec on Embedded Devices With CHACHA20-POLY1305

tl;dr: IPSec at ~400 Mbps on a CPU with no hardware crypto acceleration.

My DIY switch/firewall has an Intel Celeron J1900. While it's quite fast at forwarding/routing packets (>1Gbps tested/confirmed, likely actual ~3.6 Gbps), it can barely do 25 Mbps with strongSwan's defaults. The main reason is that it's lacking AES-NI, or any hardware crypto accelerator.

I'm assuming you've already used strongSwan before or have read another tutorial. Only extra software dependency for Ubuntu 16.04 is:

sudo apt-get install libstrongswan-extra-plugins

Config

/etc/ipsec.conf (intel.example.com)

config setup
        uniqueids=yes
        charondebug="ike 1, knl 1, cfg 1, net 1, esp 1, dmn 1,  mgr 1"

conn %default
        fragmentation=yes
        rekey=yes
        dpdaction=clear
        keyingtries=%forever
        keyexchange=ikev2
        compress=no
        dpddelay=600s

        # IKEv2 speed isn't a problem.
        ike=aes128gcm16-prfsha512-ecp521!

        # I'm not sure if ecp192 is the best choice, ask a real crypto person for verification.
        esp=chacha20poly1305-ecp192!

        authby=secret

conn pulse
        # Due to peering, these two ISPs are faster to connect via IPv4 =( 
        left=%any4
        [email protected]
        leftauth=secret
        # 2001:db8:200::/40 == Random IPv6 address. Not really needed here.
        # 10.10.10.10/32 == Random private IPv4 address.
        # 2001:db8:ff00:16b::/64 == All of our public addresses that we want to tunnel instead.

        leftsourceip=2001:db8:200::,10.10.10.10
        leftsubnet=2001:db8:100::/40,10.10.10.10/32,2001:db8:ff00:16b::/64

        right=pulse.example.com
        [email protected]
        rightauth=secret

        rightsubnet=2001:db8:ff00::/40,203.0.113.0/20,10.137.144.17/16
        # 2001:db8:ff00::/40 ==  All of their IPv6 public addresses that we want to tunnel.
        # 203.0.113.0/20 == Their public IPv4 network
        # 10.137.144.17/16 == Their private IPv4 network.

        auto=start

/etc/ipsec.conf (pulse.example.com)

config setup
        uniqueids=yes
        charondebug="ike 1, knl 1, cfg 1, net 1, esp 1, dmn 1,  mgr 1"

conn %default
        fragmentation=yes
        rekey=yes
        dpdaction=clear
        keyingtries=%forever
        keyexchange=ikev2
        # compress=no
        dpddelay=60s
        authby=secret

conn intel
        left=2001:db8:ff00::1,203.0.113.0
        [email protected]
        leftauth=secret
        leftsubnet=2001:db8:ff00::/40,203.0.113.0/20,10.137.144.17/16

        [email protected]
        rightauth=secret
        rightsourceip=10.10.10.10,2001:db8:200::
        rightsubnet=2001:db8:100::/40,10.10.10.10/32,2001:db8:ff00:16b::/64

        ike=aes128gcm16-prfsha512-ecp521!
        esp=chacha20poly1305-ecp192!

        compress=no
        dpddelay=600s

        auto=add

/etc/ipsec.secrets (intel.example.com)

@intel.example.com @pulse.example.com : PSK "Nj3eOavMxgXQX3gA1mPXy"

/etc/ipsec.secrets (pulse.examples.com)

@pulse.example.com @intel.example.com : PSK "Nj3eOavMxgXQX3gA1mPXy"

/etc/strongswan.conf (both)

We have to send the vendor ID because of CHACHA20-POLY1305.

charon {
        load_modular = yes
        plugins {
                include strongswan.d/charon/*.conf
        }
        send_vendor_id = yes
}

include strongswan.d/*.conf

Parallelizing (intel.example.com)

First, make sure you do have kernel support for chacha20.

dave@intel:~$ grep -E 'driver.*chacha20' /proc/crypto
driver       : seqiv(rfc7539esp(chacha20-simd,poly1305-simd))
driver       : rfc7539esp(chacha20-simd,poly1305-simd)
driver       : chacha20-simd
driver       : chacha20-simd
driver       : chacha20-generic

Looks good! You'll see an error message, but don't worry, it actually worked.

dave@intel:~$ sudo modprobe tcrypt alg="pcrypt(rfc7539esp(chacha20,poly1305))" type=3
modprobe: ERROR: could not insert 'tcrypt': Unknown symbol in module, or unknown parameter (see dmesg)
dave@intel:~$ grep -E 'driver.*pcrypt.*chacha20' /proc/crypto
driver       : pcrypt(pcrypt(rfc7539esp(chacha20-simd,poly1305-simd)))
driver       : pcrypt(rfc7539esp(chacha20-simd,poly1305-simd))

Parallelizing (pulse.example.com)

We're not going to enable pcrypt on pulse.example.com, since it's a single thread VM.

Testing

Run sudo systemctl restart strongswan.service on both machines first to make sure the changes have been loaded.

dave@pulse:~$ while sleep 1s; do iperf3 -s -B 2001:db8:ff00::1; done
dave@intel:~$ iperf3 -c 2001:db8:ff00::1 -R -Z -t 120 -P 128 -O 60 -B 2001:db8:100::
....
[SUM]   0.00-119.97 sec  5.60 GBytes   401 Mbits/sec  149461             sender
[SUM]   0.00-119.97 sec  5.60 GBytes   401 Mbits/sec                  receiver

iperf Done.


TODO: Get this up to >750Mbps.

Update #1: Not sure why, but I'm getting quite a bit of packet loss compared to AES128.

Sunday, September 3, 2017

Beyond Gigabit (On 1Gb Ethernet) - VLAN Bonding

Congratulation, you've got a personal gigabit internet pipe! Just plug your server into the ISP's modem and you'll get 1Gbps, right?

Nope.

You're realistically looking at ~925-950Mbps at the network layer, as 1GbE is only at the data link layer.

There's two solutions: upgrade to 10Gb hardware, or use bonding. The former is going to be a hard sell to your ISP when you're only paying for 1Gbps to begin with, so we're left with bonding as the remaining choice. (And if you're using 10GbE, you probably still want bonding for redundancy.)

Network Layout


Switch

In my case I'm using VID 2372 and 2662 for the two WAN ports.


Don't put the two WAN ports on the same VLAN ID, unless you feel like testing your modem's loopback detection.

/etc/network/interfaces

auto lo
iface lo inet loopback

auto enp1s0
iface enp1s0 inet manual
bond-master bond0

auto enp2s0
iface enp2s0 inet manual
bond-master bond0

auto enp3s0
iface enp3s0 inet manual
bond-master bond0

auto bond0
iface bond0 inet manual
bond-mode balance-rr
# bond-mode 802.3ad
bond-slaves enp1s0 enp2s0 enp3s0
bond-miimon 100
bond-downdelay 200
bond-updelay 200
# bond-lacp-rate 1
bond-xmit-hash-policy layer2
# bond-xmit-hash-policy layer3+4
hwaddress ether 00:20:91:80:23:AD

# WAN-Port-01
auto vlan2372
iface vlan2372 inet manual
bond-master vbond0
vlan-raw-device bond0
hw-mac-address 00:20:91:23:72:00

# WAN-Port-02
auto vlan2662
iface vlan2662 inet manual
bond-master vbond0
vlan-raw-device bond0
hw-mac-address 00:20:91:26:62:00

auto vbond0
iface vbond0 inet dhcp
bond-mode balance-rr
bond-slaves vlan2372 vlan2662
# bond-xmit-hash-policy layer3+4
hwaddress ether 00:20:91:15:9B:CD


Hashing

Layer 3+4 hashing is not needed as we're connecting to two physically different ports on the modem. It probably wouldn't hurt to use though.

Static MAC Addresses

I would recommend assigning a MAC address, otherwise it will be confusing to tell if the bonding is working, or if the system is just using the first NIC by itself (e.g. in PXE). Also, if the order of network cards change, the bonded interface will default to a different MAC address.

LACP

If possible, try using 802.3ad (LACP) instead of round-rr. (My switch does not load balance properly, and ends up clustering all the incoming traffic to a single port in a LACP group.) If a PEBKAC cabling incident happens, you'll only end up spamming LACPDU frames to the wrong port.

/etc/sysctl.conf

Apparently there's a "known bug" in the Linux kernel with multiple bonded links and IPv6, according to a random person on IRC. (This report is the best reference I could find.)

net.ipv6.conf.vbond0.accept_dad=0
net.ipv6.conf.vlan2372.use_tempaddr=0
net.ipv6.conf.vlan2372.autoconf=0
net.ipv6.conf.vlan2662.use_tempaddr=0
net.ipv6.conf.vlan2662.autoconf=0

I'm also turning off SLAAC on the VLANs, since I don't want to accidentally use them instead of the bonded interface.

Results

dave@intel:~$ iperf3 -c iperf.he.net --zerocopy --parallel 8 --reverse --omit 30 -t 60 --format m --interval 60
Connecting to host iperf.he.net, port 5201
Reverse mode, remote host iperf.he.net is sending
[  4] local 2a07:1c44:14b4:: port 58112 connected to 2001:470:0:238::2 port 5201
[  6] local 2a07:1c44:14b4:: port 58114 connected to 2001:470:0:238::2 port 5201
[  8] local 2a07:1c44:14b4:: port 58116 connected to 2001:470:0:238::2 port 5201
[ 10] local 2a07:1c44:14b4:: port 58118 connected to 2001:470:0:238::2 port 5201
[ 12] local 2a07:1c44:14b4:: port 58120 connected to 2001:470:0:238::2 port 5201
[ 14] local 2a07:1c44:14b4:: port 58122 connected to 2001:470:0:238::2 port 5201
[ 16] local 2a07:1c44:14b4:: port 58124 connected to 2001:470:0:238::2 port 5201
[ 18] local 2a07:1c44:14b4:: port 58126 connected to 2001:470:0:238::2 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-60.00  sec  1.02 GBytes   146 Mbits/sec
[  6]   0.00-60.00  sec  1.02 GBytes   146 Mbits/sec
[  8]   0.00-60.00  sec  1017 MBytes   142 Mbits/sec
[ 10]   0.00-60.00  sec  1.00 GBytes   143 Mbits/sec
[ 12]   0.00-60.00  sec  1.03 GBytes   148 Mbits/sec
[ 14]   0.00-60.00  sec  1.03 GBytes   148 Mbits/sec
[ 16]   0.00-60.00  sec  1.04 GBytes   149 Mbits/sec
[ 18]   0.00-60.00  sec  1.05 GBytes   151 Mbits/sec
[SUM]   0.00-60.00  sec  8.20 GBytes  1174 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-60.00  sec  1.02 GBytes   147 Mbits/sec  703             sender
[  4]   0.00-60.00  sec  1.02 GBytes   147 Mbits/sec                  receiver
[  6]   0.00-60.00  sec  1.02 GBytes   147 Mbits/sec  823             sender
[  6]   0.00-60.00  sec  1.02 GBytes   147 Mbits/sec                  receiver
[  8]   0.00-60.00  sec  1019 MBytes   142 Mbits/sec  609             sender
[  8]   0.00-60.00  sec  1020 MBytes   143 Mbits/sec                  receiver
[ 10]   0.00-60.00  sec  1.00 GBytes   143 Mbits/sec  1455             sender
[ 10]   0.00-60.00  sec  1.00 GBytes   144 Mbits/sec                  receiver
[ 12]   0.00-60.00  sec  1.03 GBytes   148 Mbits/sec  798             sender
[ 12]   0.00-60.00  sec  1.03 GBytes   148 Mbits/sec                  receiver
[ 14]   0.00-60.00  sec  1.03 GBytes   148 Mbits/sec  1233             sender
[ 14]   0.00-60.00  sec  1.03 GBytes   148 Mbits/sec                  receiver
[ 16]   0.00-60.00  sec  1.04 GBytes   150 Mbits/sec  575             sender
[ 16]   0.00-60.00  sec  1.04 GBytes   150 Mbits/sec                  receiver
[ 18]   0.00-60.00  sec  1.06 GBytes   151 Mbits/sec  577             sender
[ 18]   0.00-60.00  sec  1.06 GBytes   151 Mbits/sec                  receiver
[SUM]   0.00-60.00  sec  8.21 GBytes  1175 Mbits/sec  6773             sender
[SUM]   0.00-60.00  sec  8.22 GBytes  1177 Mbits/sec                  receiver

iperf Done.

tl;dr: 1177Mbps through a 1GbE modem.