Manual firmware upgrade of lightweight access point

The Cisco lightweight access point managed by the wireless controller is not a new product. It has been on the market for years, evolved from LWAPP to CAPWAP model, is supporting a wide range of access point models. The idea is that access point upgrades its firmware when it registers to WLC. Sounds easy? Yes. Is it working? Yes, usually, but not always. Sometimes you need to perform a manual lightweight access point firmware upgrade procedure which is not well documented. And you need to know a hidden command. I will show you how.
My lightweight access point image is too old
What are the symptoms that you cannot upgrade your access point automatically? On the web dashboard, you will see the access point reappearing with Operational Status set to Downloading. However, this does not answer the question what exactly the problem is. More answers you will get from the access point console
*Dec 17 16:39:40.227: %CAPWAP-5-SENDJOIN: sending Join Request to 172.16.1.205perform archive download capwap:/ap3g2 tar file *Dec 17 16:39:40.251: %CAPWAP-6-AP_IMG_DWNLD: Required image not found on AP. Downloading image from Controller. ERROR: Image is not a valid IOS image archive. Download image failed, notify controller!!! From:8.0.110.0 to 0.0.0.0, FailureCode:3
The access point joined the controller successfully and attempted to download the current archive from the controller, but the whole operation failed. The FailureCode:3 does not point to a reason.
My access point out-of-box had preinstalled firmware 15.3(3)JA1, the latest vWLC version 8.6.101.0 requires the 15.3(3)JG1. As we can see the 15.3(3)JA1 is compatible with controller firmware release 8.0.110.0 – both released in 2014.
In my case, I suspect one of the possible reasons why automatic firmware upgrade did not work is that for Cisco Aironet 3700 with release 15.3(3)JF has changed the platform code from ap3g2 to c3700. I could see this when I listed available images in WLC
(Cisco Controller) >show ap bundle all Primary AP Image Size Supported AP's ---------------- ---- ------------ ap1g1 13320 AP700 ap1g3 15360 AP1530 ap1g4 28452 AP1850/1810 ap1g5 24992 AP1815,1540 ap3g3 47184 AP2800,3800,1560 c1570 13040 AP1570 c3700 14340 AP1700,2700,3700
Another possible reason was that the download or image verification process changes significantly and access point was not able to verify the new image. That is not an uncommon problem in Cisco; we saw this on ASA already.
Manual firmware update
You can perform manual firmware upgrade in such situation loading either up-to-date version or any close to this. You can download the firmware from cisco.com, but you need valid service contract assigned to your CCO profile. I got 15.3(3)JF release.
First, put the downloaded archive on the server accessible from subnet where access point gets its IP address from. This image needs to be accessible via HTTP, HTTPS, FTP, TFTP or SCP.
To download, extract and install new firmware you need to use the archive command, but it is not available by default
AP84b8.02aa.bbcc#arch? % Unrecognized command
To enable the archive command, you need first enable hidden debug mode
AP84b8.02aa.bbcc#debug capwap console cli This command is meant only for debugging/troubleshooting Any configuration change may result in different behavior from centralized configuration. CAPWAP console CLI allow/disallow debugging is on
Now while the archive command is active, you can download firmware manually from your server
AP84b8.02aa.bbcc#archive download-sw tftp://192.168.1.250/ap3g2-k9w8-tar.153-3.JF.tar
The access point running old firmware will constantly attempt to join the controller and download the firmware. If you try to run manual upgrade at this point, you will see the error message.
Unable to create temp dir "flash:/update" Download image failed, notify controller!!! From:8.0.110.0 to 0.0.0.0, FailureCode:7
You can add /overwrite option to remove old firmware if you want. When the installation is completed, you need to reload the access point manually. Now it should be able to join the controller and download the latest 15.3(3)JG1 firmware from it.
Pingback: DTLS 1.2 and Cisco LWAPP / CAPWAP APs: On shooting yourself in the foot