Filter results by

Proxy Servers and Certificates

Are you blocked from installing packages using dnf or downloading files using the curl command because of the proxy server in your organization?

In the corporate environment, IT departments rely on proxy servers to bolster the security of communications with the outside world. Dealing with this inner protocol can be a difficult task. You'll need to:

  • Set system-wide or local environment variables to specify not only your proxy address and port, but also your own user name and password.
  • Append the proxy server certificate to the certificate bundle that each particular program is using.
  • Put in certain program-specific keywords to tell the program it is going through a proxy server.

Here we may not have all the answers, but we'll try to point you in the right direction. You'll also want to refer to AN100 – Enabling ARTIK Internet Access Through Firewalls – for an overview of typical firewall environments.

Overview

Getting everything right to go through a corporate proxy server is a challenge. Here's an overview – we explain the details in the sections we link to. The procedure works for us but may not for you; adapt it to your situation as needed.

  1. Set up Wi-Fi. Here you'll be specifying the Wi-Fi network (SSID) and its password, but not any proxy login information just yet.

  2. Edit your Connection Manager /etc/connman/main.conf file to add p2p to its blacklist. Otherwise you'll find that your transfers get cut off after several seconds.

  3. Get secure dnf transfers working by attaching your company's unique proxy certificate to the rest of the bundle.

    1. Determine your proxy server address.

    2. Following Specifying the Proxy Server, put your proxy name and address in /etc/profile as noted. This is also where you enter username:password as assigned to you to get through the "inner protocol" of the company proxy.

    3. Copy your local issuer certificate – the single one that comes from your IT department – to your ARTIK board root. Here we'll call it localcert.pem

    4. Go to the certificate storage directory
         cd /etc/pki/ca-trust/extracted/pem

      and append localcert.pem to the tls-ca-bundle.pem file.
         mv tls-ca-bundle.pem tls-ca-bundle.orig
         cat tls-ca-bundle.orig localcert.pem > tls-ca-bundle.pem

    Getting this far should allow dnf access through the proxy, avoiding "local certificate issuer" failures, but may not be enough for curl calls.

  4. Get secure npm and curl transfers working by giving them the certificate bundle they're looking for (but different from what dnf wanted).

    1. Copy down an up-to-date ca-bundle-crt certificate bundle.
    2. On the command line, execute
         npm config set strict-ssl false
      to prevent npm from failing due to the issuer certificate check (not the same as the "local issuer certificate" failure above).

    3. They should take your login credentials automatically from the export values you assigned above, but you can read here if you are concerned about that.

Your ARTIK board should now act like it's connected directly to the outside world (except, of course, for the URLs your IT department blocks on purpose).

Determining the Proxy Server Address

Your corporate IT department will be able to provide you with the address of your company's proxy server, but you can also find it in your desktop PC network configuration.

For a Windows® system, it can be found under:
  Start > Control Panel > Network and Internet > Internet Options
where you'll select the Connections tab and click LAN settings. You'll see the "Use automatic configuration script" file name that the IT department has specified.

Copy the full line (make sure you have the hidden parts highlighted too), paste it into your browser, and download the proxy.pac file. Open this text file, and you'll find the same "default" address over and over – that's the address you'll use below.

Specifying the Proxy Server

Our information here provides tested instructions only regarding http and https traffic through the proxy server; other traffic may require additional considerations.

Standard Proxy Variables

Linux has standardized the http_proxy and https_proxy environment variables to specify the proxy server address; ftp_proxy is also used. You can enter these on the command line:

 export http_proxy="http://username:password@123.456.789.200:8080/"
 export https_proxy="http://username:password@123.456.789.200:8080/"
 export ftp_proxy="http://username:password@123.456.789.200:8080/"

before starting wpa_supplicant, or you could copy them into your /etc/profile, ~/.bashrc, or other start-up script to make them a permanent part of your boot procedure. Type export at the Linux command prompt to see them echoed and verify that they are correct.

Do you have any special characters like "!" in your password? Prefix each with a backslash (\), or substitute its ASCII code (e.g. replace "@" by "%40"). Still getting rejected? Reset your password to use underscores instead (since they do not require the backslash prefix).

Non-Standard Proxy Variables

Not every program recognizes the Linux standard environment variable proxy names. Some programs use an association specified in their configuration file, and it may not be the same variable name. For example, npmuses this syntax:

npm config set https-proxy http://proxy.company.com:8080

(https-proxy instead of https_proxy) to set the variable. Refer to the sections below for more information.

Connection Manager (connman)

ARTIK images released February 2016 and later rely on Connection Manager to set up network services. connman can have a side effect when working through proxy servers, disrupting service ("host unreachable" or "no route to host" errors) within a minute or so after boot. To disable connman:

  1. systemctl disable connman.service
  2. reboot

On subsequent booting you need to manually enter the following commands:

  1. systemctl start wpa_supplicant
  2. dhclient wlan0

Refer to the Wi-Fi article for more details on connman.

Certificates

An SSL/TLS-secured network connection requires each client to have a local copy of the server certificate that can be compared to the certificate that the server transmits to establish the connection, as we did here when we set up our own private server.

Where to Find the Certificate Bundle

Different programs look for a certificate bundle in various locations. For example:

  • If you look under /etc/ssl/certs, you'll find a certificate bundle.

  • When you look closer by listing with ls -al, you'll see that the bundle is actually a symbolic link to /etc/pki/ca-trust/extracted/pem.

  • If you set up your own independent server-client link (as we did in the tutorials), you may have placed your mini-CA certificate in /etc/pki/tls/ .

There are too many variations to cover them all here – search online for your specific needs.

Most schemes look for all certificate packages and extensions that happen to be in the directory of interest, although some expect a particular file name or extension type.

Updating ca-bundle.crt (curl and npm)

The certificate bundle described here is used by curl as well as by other programs, such as npm.

Go to the /etc/pki/ca-trust/extracted/pem directory and you will see several .crt files in standard PEM encoding.

While in that directory, run this command.

curl https://curl.haxx.se/ca/cacert.pem -o ca-bundle.crt

The addition of this certificate bundle should satisfy curl and npm requirements.

You could run into a 'chicken or egg' problem here, where you cannot download a new certificates file because you do not have current certificates! If download fails, try again adding
  --insecure
to the command.

Local Issuer Certificate

With an intermediate proxy server involved, the certificate situation becomes even more complicated, as the client must also have a copy of the proxy server certificate. You'll need to append this "local" certificate to the end of your existing certificate bundle.

When you see an error message about a "local issuer certificate" not being found, "local" usually refers to a missing or incorrect proxy server certificate.

You can obtain the local issuer certificate from your IT department, or possibly just export it from your PC's certificate store. You then append it using a method of your choice – for example:

cat bundle.crt localcert.crt >> new-bundle.crt

would add the indicated local cert file to an existing bundle.crt file; you would then delete the old bundle file and rename the new one to replace the old one.

Knowing when you need to append a local certificate, versus simply including it as a separate file, may not be obvious. If all else fails, add your proxy certificate to each bundle you find in any location, and revert if the addition causes security failures.

Program-Specific Syntax

DNF

dnf uses the Linux standard proxy export variables. You'll find the configuration file at /etc/dnf/dnf.conf if you need to specify a unique proxy server for dnf. It uses the same names that the Linux standard does.

NPM

Before attempting any npm operations, make sure you have an updated certificate file.

Early versions of npm required the proxy to be specified in the local configuration file .npmrc. Later versions accept the Linux standard proxy export variables. You can also use the configuration file to specify it, but it uses a different naming convention: note the use of a hyphen instead of an underscore.

npm config set https-proxy http://proxy.company.com:8080

The command sets the related value in the .npmrc file.

Certain packages loaded through npm are notoriously difficult to send through a proxy server because their dependencies do not follow the standard. Sometimes it's easier (although not recommended) to just configure npm to not demand secure transfers:

npm config set strict-ssl false

and then set it to pull from its unsecure http registry, instead of https:

npm config set registry "http://registry.npmjs.org/"

After the commands above, the .npmrc file contents are now:

  https-proxy=http://proxy.company.com:8080/:
  registry=http://registry.npmjs.org/
  strict-ssl=false

If you are getting node-gyp folder access errors, try putting --unsafe-perm on the command line. You can specify --verbose for troubleshooting. You'll typically want to use -g for global.

Even with all these workarounds, some older npm packages you might want to use may still not install correctly. At a certain point, you may have to just give up and install them outside of a proxy server environment.

CURL and WGET

The curl and wget commands have their own unique syntax for specifying proxy address/username/password information separately from that of the remote site. Search online for more information.

curl --proxy <[protocol://][user:password@]proxyhost[:port]> -L <http...>

The configuration for wget is found in the /etc/wgetrc file, but is usually not needed as it uses the same environment variable names found in /etc/profile. It may be necessary to add --no-check-certificate to the wget command line to bypass a proxy (that way you will get a WARNING instead of an ERROR for 'unable to get issuer certificate').

Refer to the Certificates section for curl certificate location and update information.

Arduino IDE

The Arduino IDE (running on your host PC) needs to be set to allow it to download libraries and updates. Under File > Preferences you'll find the Network tab to enter the proxy.pac location name that you found earlier (you don't need to enter the numeric proxy address itself, just the path and file name as shown above).

On your ARTIK board: The Arduino communications setup routine (the curl command shown here) properly accesses the Linux standard proxy export variables when present.

Once you have that software installed, you can go through the proxy using the Arduino IDE to send files to the ARTIK board for execution (as shown here).

If you are using Arduino sketches to communicate with ARTIK Cloud, use Web REST protocols instead of MQTT. The currently available Arduino network drivers are unable to connect through an authenticated proxy server to ARTIK Cloud over MQTT.

Mosquitto and Node-RED

It is relatively easy to set up Mosquitto and/or Node-RED for SSL/TLS operation. Refer to the Secure Links article for details.

Node-RED running on an ARTIK module connects through an authenticated proxy server to ARTIK Cloud over MQTT without any issues, once properly configured.

Mozilla Firefox

You may be using an X11 GUI as discussed in the Displays article. You will need to use the Firefox GUI to add the local proxy address and server certificate. It may not take this information automatically from the Linux environment.

For the proxy:
   Preferences » Advanced » Network » Settings
You should be able to select "Use system proxy settings" but may need to make manual entries instead.

For the certificate:
   Preferences » Advanced » Certificates » Authorities » Import
Select the local issuer certificate (you may have saved it on your ARTIK root drive here.

Last updated on: