AC Controller: ESP8266 Telnet Server for logs

Since the start of the project I kept thinking I needed a way to be able to get logs from the system remotely. I originally considered some remote server, maybe use adafruit.io service to push error codes with mqtt, which will allow me easy remote visualization and analysis. I am still considering that option, but as I have already expressed, I hate depending on services provided by thirdparties, so even if I end up taking that path, I decided to add a telnet server to the ESP8266 and send the logs there.

For those not familiar with Telnet, it is just a TCP socket in port 23 where you send ASCII data... super simple... can not get simpler than that... I found this example: https://github.com/esp8266/Arduino/tree/master/libraries/ESP8266WiFi/examples/WiFiTelnetToSerial doing telnet to serial and pretty much reused that.

This is what I did. At the declarations I added:

1. WiFiServer telnet(23);
2. WiFiClient telnetClients;
3. int disconnectedClient = 1;
4. 
   
5. #define DEBUG_LOG_LN(x) { Serial.println(x); if(telnetClients) { telnetClients.println(x); } }
6. #define DEBUG_LOG(x) { Serial.print(x); if(telnetClients) { telnetClients.print(x); } }

Basically for declaring the server (line 1), the client (line 2) and a pair of macros for sending logs to serial only or to serial and telnet if there is a telnet client.

In setup:

1.   telnet.begin();
2.   telnet.setNoDelay(true);
3.   DEBUG_LOG_LN("Telnet server started");  

Start the telnet server (line 1), enable no delay (line 2) which disables the naggle algorithm that would buffer data before pushing it out on the TCP socket.

In loop:

1.   if (telnet.hasClient()) {
2.     if (!telnetClients || !telnetClients.connected()) {
3.       if (telnetClients) {
4.         telnetClients.stop();
5.         DEBUG_LOG_LN("Telnet Client Stop");
6.       }
7.       telnetClients = telnet.available();
8.       DEBUG_LOG_LN("New Telnet client");
9.       telnetClients.flush();  // clear input buffer, else you get strange characters 
10.       disconnectedClient = 0;
11.     }
12.   }
13.   else {
14.     if(!telnetClients.connected()) {
15.       if(disconnectedClient == 0) {
16.         DEBUG_LOG_LN("Client Not connected");
17.         telnetClients.stop();
18.         disconnectedClient = 1;
19.       }
20.     }
21.   }

This will connect to a telnet client if there happens to be one available (lines 1 to 12) and will disconnect the client if the connection gets dropped (lines 14 to 21). With this code, if a client comes in, the current client connection gets dropped (lines 3 to 6).

Lastly, you may notice that I replaced all the Serial.print and Serial.println with DEBUG_LOG and DEBUG_LOG_LN. Those 2 macros are the ones pushing the logs to serial or both to serial and telnet. I will probably extend those in the future to add line number and time.

As with the whole project, the work is being checked in here: https://github.com/desordenado77/esp8266_netatmo_acController

AC Controller: ESP8266 Netatmo Thermostat control

I happen to own a Netatmo thermostat. The company that supplies the gas for my boiler was nice enough to give it to me as a present this winter. The product is an internet controlled thermostat. It is OK, would recommend it to someone looking for that type of product. Personally I hate the fact that I am using a product which will stop working if the company that supplies the cloud service disappears, or if they decide to start charging a monthly fee, but I got it for free, so I am getting to enjoy it without having to think about those issues... hehehe...

Netatmo is also nice enough to give an SDK to be able to access its devices (up to certain limits, of course), but enough for my needs. So, when I started to think about my AC controller, I came up with the idea of using Netatmo thermostat as my remote control and sensor. I first did some investigation of their SDK API using python scripts which I will check in some day (right now they have my personal information hardcoded, so I need to do some clean up). The process to be able to use the SDK goes through creating a new application in your user area in their web page. That gives you some IDs you will need to use later for identification.

Once I manged to get passed the authentication I met my first issue, the temperature is only updated once an hour. That sucks. Not that I was planning on checking the temperature every minute, but at least 4 times an hour. Doing some research on their forum, I found an undocumented API called "syncthermstate" which will force the thermostat to update the cloud with the current sensor values. This API requires "write_thermostat" scope to be enabled during authentication. I also found that once I get an authentication token and a refresh token, I could just use the refresh token to get new authentication tokens at any time, not requiring me to give my username and password again, which simplifies things. So I managed to get a python script with a bunch of https requests talking with the netatmo cloud and getting sensor readings on request from my thermostat.

My ESP8266 board arrived and the time had come to start porting my python script to Arduino. Turns out https is vaguely supported. There is an https example in ESP8266Wifi (does not look like the place for an https client to be for me, but hey, who am I to judge), here: https://github.com/esp8266/Arduino/tree/master/libraries/ESP8266WiFi/examples/HTTPSRequest
And that seems to be the best example. Issue with that example is that everything is hardcoded, but I used that as a building block for my work. Basically you need to write the whole headers and parse the results. To do that I enabled debugging in my python script (which was using "requests" for the http client) by adding the following in the imports:


import logging
try:
    import http.client as http_client
except ImportError:
    # Python 2
    import httplib as http_client

And this in the code:

http_client.HTTPConnection.debuglevel = 1
logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True

That showed me the headers as being created by the python "requests" and as received from Netatmo, so I recreated the same thing in Arduino.

One thing I noticed when I managed to send the right headers from Arduino to netatmo was that the reply from Netatmo was using chunked encoding. Chunked encoding is a tool from HTTP 1.1 to packetize the responses. Since I had to write the parser for the responses, I decided to force the server not to use chunked encoding by telling it that the ESP8266 is an HTTP 1.0 client instead of 1.1 (chunked transfer encoding is not supported in 1.0).

You may find the code with all of this in here:  https://github.com/desordenado77/esp8266_netatmo_acController

Right now it does not do any control, it is just the building blocks on top of which I will be doing my final application.

Check httpsPostRequest on how am I doing my POST requests. As you can see my parser of the response is very simple and most likely not error proof. I am also not doing any certificate verification for now, which could potentially be susceptible of a man in the middle attack (somebody impersonating Netatmo servers).

In getThermostatData I am synchronizing my thermostat with the cloud and then reading the temperatures. If any of the calls to Netatmo servers fails I attempt to get a new authentication token using my refresh token.

Hope that helps someone... I will continue working on this project, the things to add next are:
- Real temperature check and AC control.
- Add remote logging capabilities
- LED notification
- Physical control with a button (using short press, long press, very long press for different actions)

AC Controller: ESP8266 WiFi Relay

Long time no write... been busy... but I am back...
This June we had a huge rise of temperatures and I suffered my crappy AC thermostat, which does not seem to measure temperature correctly and has no programability. I needed some fix for that. I could have bought a new thermostat, but that would have been very easy. So I decided to get myself an esp8266 board with a relay so that I could switch my AC on an off. I already have a nice wifi thermostat to control my boiler and the company making that one (netatmo) is nice enough to provide an SDK to access the thermostat data, so my plan is to try to use that as input to this ESP8266 board to turn on and off the AC. Here goes my journey to try to achieve such goal...

I first bought the ESP8266 board with a relay pictured above these lines. On ebay. Less than 9 euros delivered. It was my first go at ESP8266, but I had read that you could use the Arduino IDE to program them, so what could go wrong?

I had to solder the 3 pin serial port and the 2 pin boot mode selector, other than that the board was ready to be used. As for setting up the Arduino IDE, everything seems well explained here: https://github.com/esp8266/Arduino

Install Arduino IDE (1.8.3 in my case)
Open Preferences window (File->Preferences).
Enter http://arduino.esp8266.com/stable/package_esp8266com_index.json into Additional Board Manager URLs field. You can add multiple URLs, separating them with commas.
Open Boards Manager from Tools > Board menu and install esp8266 platform (and don't forget to select your ESP8266 board from Tools > Board menu after installation).
Select the Generic ESP8266 board
Select the device used for serial connection (you need a 3V3 232 adapter)
I opened the Blink Sketch and replaced the GPIO used with number 2, which corresponds to the blue LED mounted on the ESP8266 board. Verify, Upload and got a blinking LED... ready to start the real work

Ibanez TSA5TVR

Recently bought this cute retro-looking amp, Ibanez TSA5TVR. It is a 5Watt, all valve, class A amp, ideal for home use. I am not good at describing tone, for me it is the same as it happens with the wines: if the wine is good I usually like it and if its bad I normally don't. So this amp I like, so I suppose it is good... that is all I will say about it... check other places for good reviews. I will discuss here about some "non intrusive" modifications that I did.

The amp has a headphone output, but somehow the amp does not switch off the speaker when you plug the headphone and you are not supposed to unplug the speaker when the amp is working or you could damage the transformer (or so I have read). So what is the solution? my first thought was to replace the speaker with an 8 ohm dummy load. Doing some more research I found about L-Pad attenuators. This is kind of like a rotary potentiometer that lets you change the output level on the speaker with the peculiarity that the amp always "sees" the same 8 ohm loads no matter the configured level. This basically allows me to reduce the output level of the amp to 0, so that the speaker is muted, but still get audio in the headphone output. Some people also use it as a way to be able to crank the power amp but still not annoy the neighbors. I have not tried this yet. There is more info on L-Pads and also a calculator here. I added a bypass switch (to quickly switch between attenuator and normal) and some people also adds a switch to get more brightness, see here. I would have liked to add this to the amp instead of making it a external box, but I don't want to void the warranty, so it will stay in a box for 2 years, I suppose.

The other thing I did was a footswitch with LED to turn on/off the included overdrive. I tried the Marshall schematic that I found on the internet... did not work. This is the circuit:



I looked for other circuits and found this other one which worked OK:



Not having the schematic of the ibanez IFS1G original footswitch from Ibanez, I can not tell if this is the exact same circuit, but this does indeed work. I used a 1K resistor for R1, but I suppose you can adjust that depending on the desired brightness of the LED.

And that is all for now... further work on the amp will most likely happen when the warranty is over, until then I will have to hold myself from getting my dirty soldering iron inside it.

Sooperlooper

I have been playing with SooperLooper, a pretty cool and simple looper application for Linux and Mac. I use ubuntu and installation there is simple, it is available in the software center if you use the gui or with sudo apt-get install sooperlooper if you are more hardcore. The application has 2 parts, a GUI and the server which communicate with each other using OSC, the standard aiming to replace Midi. You can run the server only with the command sooperlooper and you can run the GUI+server with slgui. 

My first tests were done with the GUI. You need to run slgui and make the right connections in JACK. I use QJackCtl and this are the connections I set up for redirecting audio input to slgui capture and its output to the audio output:

I am interested on running the looper headless, so I have been investigating on how to send OSC commands from the command line. I installed pyliblo-utils: sudo apt-get install pyliblo-utils which adds a tool called send_osc and dump_osc to send and receive OSC messages. Once I did that I opened 3 terminals running the following things on each:

Terminal 1: run sooperlooper (without GUI), this is listening OSC commands in port 9951 by default
Terminal 2: run dump_osc to receive OSC commands at port 9952: dump_osc 9952
Terminal 3: this is where things get interesting:
First I run the ping command:
send_osc 9951 /ping osc.udp://localhost:9952/ osc.udp://localhost:9952/
And I get the following in Terminal 2:
osc.udp://localhost:9952/ ,ssi osc.udp://berenjena:9951/ 1.7.0 1

then I tried other commands from the sooperlooper documentation here.

Next steps: I might think of a way to interface sooperlooper from a keyboard by converting keystrokes to OSC. I also would like to see how it performs in a raspberry pi (EDIT: Seems like someone has been having similar thoughts: http://journeytounknownsoundscapes.blogspot.com.es/2015/05/a-new-looper-in-town-raspberry-pi-with.html?view=classic )

Kula6: Schematic mistake

From the beginning of this project I had a mistake in my schematic that I was too embarrassed to recognize. My AVR ADCs were not working!!! I originally thought that I had somehow broken my processor, but turns out that I was wrong. If you see my schematic in this post you will notice that the AREF is connected to ground... wrong!!! it needs to be connected to VCC (or whichever is your reference for the analog signals). Here is a picture of how the connection should be:

 

I actually added a 100nF capacitor to ground also, which is not pictured in the schematic.

I was only planning on using the ADC for debug purposes, don't have much plans on using it on the final result, but it was frustrating that I could not get it to work.

And here is the code to read from the ADC connected to ADC0 in pin40:

#include <avr/io.h>

#include <util/delay.h>

#include <avr/interrupt.h>

#include <stdio.h>

void adc_init()

{

 // Select Vref=AVcc

 ADMUX |= (1<<REFS0);

 //set prescaller to 128 and enable ADC

 ADCSRA |= (1<<ADPS2)|(1<<ADPS1)|(1<<ADPS0)|(1<<ADEN);

}

uint16_t adc_read(uint8_t ADCchannel)

{

 //select ADC channel with safety mask

 ADMUX = (ADMUX & 0xF0) | (ADCchannel & 0x0F);

 //single conversion mode

 ADCSRA |= (1<<ADSC);

 // wait until ADC conversion is complete

 while( ADCSRA & (1<<ADSC) );

 return ADC;

}

int main(void)

{

    adc_init();

    DDRC |= (1<<0);     //Set PortC Pin0 as an output

    PORTC |= (1<<0);        //Set PortC Pin0 high to turn on LED

    _delay_ms(500);

    PORTC ^= (1<<0);

    while(1) {

        uint16_t adc_result0 = adc_read(0);      // read adc value at PA0

        if(adc_result0>512) {

            PORTC |= (1<<0);

        }

        else {

            PORTC &= ~(1<<0);

        }

        _delay_ms(500);

    }

    return 0;

}

Kula6: Midi IO

I have extended my platform to include midi IO, a button and a potentiometer. Here is the schematic of my platform as it is now:

And this is how the board looks:

The code it is simple, just takes Midi input, buffers it and sends it to the output. The purpose of the code was just to test that the hardware was OK.

#include <avr/io.h>
#include <util/delay.h>
#include <avr/interrupt.h>


unsigned char data[256];
int readPtr = 0;
int writePtr = 0;
int elements = 0;

#define BAUDRATE 31250
#define UBRR 5

void uart_init(int baudrate)
{
  // calculate division factor for requested baud rate, and set it
unsigned short bauddiv = ((F_CPU+(baudrate*8L))/(baudrate*16L)-1);
UBRRL = bauddiv;
#ifdef UBRRH
UBRRH |= bauddiv>>8;
#endif

    UCSRB = ((1<<RXEN) | (1<<TXEN) | (1<<RXCIE));   // Enable Receiver, Transmitter, Receive Interrupt
    UCSRC = ((1<<URSEL) | (1<<UCSZ1) | (1<<UCSZ0));     // 8N1 data frame
}

void uart_putc(unsigned char c)
{
    // wait until UDR ready
    while(!(UCSRA & (1 << UDRE)));
    UDR = c;    // send character
}

ISR(USART_RXC_vect)
{
    unsigned char c = UDR;

    data[writePtr] = c;
    writePtr++;
    elements++;
    if(writePtr == 256){
        writePtr = 0;
    }
}

int main(void)
{
    //Setup the clock
    cli();            //Disable global interrupts
    uart_init(BAUDRATE);
    sei();            //Enable global interrupts

    while(1){
        if(elements > 0){
            uart_putc(data[readPtr]);
            readPtr++;
            elements--;
            if(readPtr>=256){
                readPtr = 0;
            }
        }
        else {
            _delay_ms(10);
        }
    }
    return 0;
}