05 Jul

Repository with Docker templates

Don't be afraid​ of the source code

Everyone who worked on any project knows that sites like GitHub, GitLab or StackOverflow are full of open-source examples, functions, issue solutions and so on. I used them as well. It is not obligatory, but I feel that it is fair if I share something I create that may be useful for others. After all, I know what I was looking for in the past. No matter if we are creating Python script, the Java application or Docker templates.

I decided to create a small project called the DockerTemplates. The repository contains at the moment docker-compose files for various products I set up using Docker containers. I prefer using the docker-compose for everything even project consists only of one container. I find it the easiest way to maintain a consistent approach across multiple systems or projects. You will find the docker-compose configuration files in folders named after the application. In each folder, there is a separate documentation of what are the configuration details implemented in each file.

The initial release of this repository contains configuration files for following applications:

  • AWX
  • Jenkins
  • GitLab-CE

The repository will accept pull requests for any registered GitLab user so it is easy to participate and put your configuration. Please check requirements described in CONTRIBUTING.md guideline. You can also email files directly to me or leave a comment.

18 Jun

Automated scripts can send commands faster than RP can process

Juniper automation with Ansible

When I was writing one Ansible playbook I faced an interesting situation. We all keep forgetting that automated tasks are executed faster than from CLI. Ane we do not take it into consideration when we write playbooks. Time is only a problem when playbook execution takes to much time.

On Juniper SRX you can enable VPN debugging using the traceoptions feature on IKE and IPSec processes. By default, it stores logs of all configured VPNs on the device in either /var/log/kmd file or one specified in traceoptions configuration. JunOS 11.4R3 brings additional enhancement to limit debugging to single VPN tunnel specified by local and peer IPs. You can turn it on by request security ike debug-enable command. This is very handy because in most cases you want to troubleshoot just the not operational connection.

There is a slight difference between the output of those commands in the log file. If you use the traceoptions you will see pretty much standard Juniper log output

[Mar 25 13:12:12]IPSec SA done callback called for sa-cfg VPN-V201 local:10.0.201.3, remote:10.0.201.4 IKEv1 with status Timed out

If you decide to perform troubleshooting using the second method you will get output like that

[Mar 25 13:20:12][10.0.201.3 <-> 10.0.201.4] IPSec SA done callback called for sa-cfg VPN-V201 local:10.0.201.3, remote:10.0.201.4 IKEv1 with status Timed out

The log now contains both peers IP addresses. It is really handy for my playbook. Because in one of the tasks I need to get specific information from the log output I can easily identify interesting lines using the peer IP addresses.

The Playbook

There are two tasks in my playbook divided by 60 seconds pause. In the first task, I send set of three commands to the router using the juniper_junos_command module here but it works the same in the one available in Ansible. I send three commands to save the time, and also provide the variables as a parameter – I gather them in previous tasks.

    - name: Prepare and start VPN debugging on remote device
      juniper_junos_command:
        commands:
          - "request security ike debug-disable"
          - "clear log VPN"
          - "request security ike debug-enable local {{ sa_tunnel_local_gateway }} remote {{ sa_tunnel_remote_gateway }}"
        format: xml

Then after one minute pause required to generate the logs I run another task where I simply read the log file via CLI command.

- name: Collect VPN logs
  juniper_junos_command:
    commands:
      - "show log VPN"
    format: xml
  register: result_VPN_log_xml

The problem

If you run those commands in this order from CLI you will get logs from troubleshooting the single VPN specified by IP parameters. But if you execute them from Ansible playbook you would not get the peer-specific logs output. It looks like the request command is ignored by the router. If you enable commands logging on the router you will see those from netconf as well. You can also check the debugging status directly from the router CLI. If no peer-specific debugging is running you will get following output

show security ike debug-status
Enabled
flag: all
level: 5

When the debug is running you will get information about its level and the peer’s addresses

show security ike debug-status
Enabled
flag: all
level: 7
Local IP: 10.0.201.3, Remote IP: 10.0.201.4

Because the delivery of command itself was not a problem then one of the workarounds could be a playbook, where I send only one command in each task. This workaround is working but it is not optimal. Remember that connection to the router is opened for each task and closed when the task completes. It takes time and the more command you want to send the longer it takes to finish the playbook.

Don’t send commands to fast

I asked JTAC for help in this case. It happened not to be standard customer problem, it was verified in their lab and at some point, the Engineering team was asked for support. I received the answer I like the best – “it is not a bug, it is a feature”!

Because there is no output when you execute any of the three commands from the first task the Ansible module sends the next command almost without the gap after the previous one. As Junos does not acknowledge that it has finished the first request command, the second comes in and gets ignored as the first one is not finished yet. Remember, you can never issue commands via CLI as fast as you can send them from the script.

I still want to send multiple commands in one task to save time, so here is other way to mittigate the problem

    - name: Prepare and start VPN debugging on remote device
      juniper_junos_command:
        commands:
          - "request security ike debug-disable"
          - "clear log VPN"
          - "show system uptime"
          - "request security ike debug-enable local {{ sa_tunnel_local_gateway }} remote {{ sa_tunnel_remote_gateway }}"
        format: xml

To slow down the task a little I put additional command before the ignored request command. It does not take much time to execute it and it generates the output that I simply ignore.

There is no documentation of how long the gap between commands should be but it would differ depending on platform.

 

06 Jun

Dynamic inventory from GIT on AWX

Blog post title - automation

In many deployments, people do not run the Ansible playbooks directly via command line. In the long term, it is not flexible and cannot provide proper permission granulation as well as trusted code control. So people use Ansible or its free version – AWX. Every playbook, no matter how you run it, needs an inventory definition. Sometimes you use static files with a list of hosts, another time you use the script that dynamically provides the inventory for the playbook. We call it then a dynamic inventory.

If you keep your Ansible project with all scripts and playbooks on GIT repository, you can import it to AWX. You can also maintain dynamic inventory scripts as a part of your project and use them to build AWX inventory.

Read More

07 May

Why I use VMware Photon OS for Docker

Photon OS Project provides lightweight container-oriented operating system you can run as VM

One of my readers asked me what platform I use for my docker containers in the lab. He assumed it is one of the public cloud platform providing such service. This is not true. I run few containers in public cloud, but this is not cost-optimal for lab tests. Especially in the way I sometimes work, when there might be few hours gap between the tests. I run all the containers locally on my own infrastructure which does not consist of few racks in mine basement. Actually, it is very small, so it has to be resource-optimal. I decided that for now, VMware Photon OS Project is the best solution for me.

Read More

26 Apr

Container names and docker-compose

This post is related to Docker and automation

You learn new things every day. Quite often by accident because something is not working as you expect or it just stopped working. This is one of those situations where I learned to be careful how to use container names using docker-compose and how containers refer to each other names. This is very important if they share the same network.

To perform some tests I created a lab where I run Ansible AWX (the free Ansible Tower version) and Netbox IPAM. I decided to use docker-compose because it gave me an easy way to manage the lab. By default, both AWX and Netbox is put in dedicated Network just for the containers of each installation. This means the isolation – the desired way you want to run containers. But in my lab, AWX need to talk to Netbox API to pull inventory. Because it is my lab I did the simplest thing.

Read More

13 Apr

Use Ansible to check SNMP on remote device

Best way to learn Ansible and the whole idea of automation is to start from the small playbooks and then grow big. If you first automate simple tasks, even those that may be easier and quicker to perform from the command line, you will learn how Ansible is working. Let’s say, we want to test if SNMP is responding on the remote host (we name it HostA). We will use SNMPv3 and authPriv security model. And of course, we want to write the Ansible playbook and run it on server HostNOC.

Read More

27 Mar

Add new interface to vSRX on VMWare Workstation

Sometimes you want to add a new interface to existing virtual router. I am sure everyone will agree with me that creating new virtual router from scratch and then moving the running configuration on it is far away from flexible approach – you want to modify existing instance. Adding new interface on vSRX on VMWare Workstation should be as simple as adding new PIC card on the physical device. There is just one thing you need to remember about.

First, you need to power off your virtual machine. At such point, I usually also make a snapshot of the VM in case of any problems. Then add new Network Interface in VM settings like for any other virtual device running on VMWare Workstation hypervisor, and assign all parameters as required. If you boot the vSRX now, you will get stuck at Wind River Linux login prompt

Wind River Linux login prompt

Wind River Linux login prompt on vSRX – remember that vSRX run as a process on Linux operating system

It is hard to guess at this point why it is not booting into JunOS. The only thing we changes is the new NIC we add. Indeed this is the source of the problem. vSRX requires NIC type defined as vmxnet3 device type, while for VMWare Workstation it does not default.

To fix this go to the folder where you store your virtual machine and find the file with .vmx extension. That is configuration file of the VM. In the end, you will see latest entries which cover newly added NIC. The index will differ depending on how many NICs you already have.

ethernet3.pciSlotNumber = "256"
ethernet3.connectionType = "custom"
ethernet3.addressType = "generated"
ethernet3.vnet = "VMnet19"
ethernet3.displayName = "VMnet19"
ethernet3.present = "TRUE"
ethernet3.generatedAddress = "00:0c:29:b1:f5:c4"
ethernet3.generatedAddressOffset = "30"

To define NIC type, you need to add one line to the configuration file

ethernet3.virtualDev = "vmxnet3"

Now vSRX will boot as expected and you will find the new interface in show interface terse output.

06 Mar

Response attributes on Synology RADIUS server

Secure Your Network

You can use Synology NAS products not only as file storage. The DSM system provides a broad spectrum of additional services via packages – either signed by Synology or provided by third-party communities – the SynoCommunity repository is the most known. The only limitation you have is your NAS hardware platform and its performance. I own DS211 model which reached its End of Sale status already. I run several network services like DNS server there, and I use it as remote storage for SMB and AFP shares as well as for iSCSI LUNs. It also provides some services for my home network and lab. One of them is RADIUS server.

I want to show you how you can use the Synology products as RADIUS server which will use LDAP database for user authentication. This way local accounts that I use for network shares authentication remains separated from any additional accounts dedicated to RADIUS service. Both RADIUS and LDAP runs on the NAS itself.

Read More

01 Feb

How to register lightweight AP in WLC using DNS records

Wireless networks

So you have your brand new shiny lightweight AP from Cisco and wireless controller. The beauty of the managed wireless solution is that it requires almost no effort to add a new access point to existing local network. There are multiple ways how lightweight AP can discover the WLC – I will show you, in my opinion, the easiest one.

Access Point registration

The WLC has two essential ports – management and service. The second one we use mostly for recovery procedure on physical appliances and is not that important when you use vWLC. The Management Port is the one you use to connect to WLC GUI. But by default, it is also used to register access point. You can always change this by selecting the “Enable Dynamic AP Management” option in other port configuration.

There are multiple options how lightweight access point can discover the controller. I think one of the most known is by using DHCP option 43, but that usually requires quite a lot of work and may not be supported by all DHCP server software. In my opinion, the easiest way to let AP discover the controller is via DNS discovery. There are two requirements for this solution to work:

  • The DHCP server must provide DNS servers IP addresses and the domain name in response
  • You need to configure CISCO-LWAPP-CONTROLLER.localdomain or CISCO-CAPWAP-CONTROLLER.localdomain where the ‘localdomain’ is the access point domain provided by DHCP.

So the whole required configuration is limited to setting up DHCP like for any other host you have in a network (no unique attributes), adding DNS entry and configure switch port in proper VLAN.

Before AP can register at WLC, we must configure the WLC itself as the master controller for the wireless network. To do that go to Controller -> Advanced -> Master Controller Mode, click the checkbox and apply the change.

When you boot up the AP, it should obtain its IP address from DHCP server. You should also see logs as below on AP console which means access point is trying to join the controller provided in DNS configuration.

*Dec 16 23:19:42.015: AP has SHA2 MIC certificate - Using SHA2 MIC certificate for DTLS.

*Dec 16 23:19:42.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 172.16.1.205 peer_port: 5246
*Dec 16 23:19:42.223: %CAPWAP-5-DTLSREQSUCC: DTLS connection created sucessfully peer_ip: 172.16.1.205 peer_port: 5246
*Dec 16 23:19:42.223: %CAPWAP-5-SENDJOIN: sending Join Request to 172.16.1.205
*Dec 16 23:19:47.223: %CAPWAP-5-SENDJOIN: sending Join Request to 172.16.1.205

It takes 1-2 minute for AP to join the controller, even more, to be operational. Usually new AP first require firmware upgrade which takes time (unless you are needed do to it manually because it is too old compared to WLC firmware which I explained in Manual firmware upgrade of lightweight access point post). When registration process completes, you will see the access point on the list in the Wireless section.

 

09 Jan

Setting up vWLC controller in 10 minutes

Wireless networks

Those are not my first steps with wireless controllers or wireless networking. I had my first touch of enterprise-class wireless networks like 7-8 years ago when Cisco WLC controllers hit the market and lead the evolving world of wireless networking. Working for Cisco Gold Partner back then gave me the opportunity to configure few both autonomous and centrally managed wireless networks at different scale. I also performed a security audit of one quite big installation. Back then there was no virtual vWLC controller back then.

Why am I getting back to the Cisco wireless solutions now? I got Cisco AIR-CAP3702 access point lately, so it was excellent opportunity to refresh my knowledge and look at the past few years changes. It is a popular enterprise model that can work either as an autonomous access point or managed via a controller. So this model is for tests and PoC labs for me, so I decided to check what has changed in Cisco wireless world.

Read More

This website stores some user agent data. These data are used to provide a more personalized experience and to track your whereabouts around our website in compliance with the European General Data Protection Regulation. If you decide to opt-out of any future tracking, a cookie will be set up in your browser to remember this choice for one year. I Agree, Deny
528