cloudClock: github repository now available

Repository is now available in https://github.com/desordenado77/cloudClock

cloudClock: ArduinoJson or python json

I am planning on having all data from the cloudClock stored in files in json format, so I looked for a json library for arduino. I found ArduinoJson, downloaded it, imported the library in the arduino 1.5 IDE and... not working... it complained about missing core-dependencies. Some research and I found the file:
~/Arduino/libraries/ArduinoJson/library.properties

Which has the following content:

name=ArduinoJson
version=4.0
author=Benoit Blanchon <http://blog.benoitblanchon.fr/>
maintainer=Benoit Blanchon <http://blog.benoitblanchon.fr/>
sentence=An efficient and elegant JSON library for Arduino
paragraph=Supports JSON parsing and formatting. Uses fixed memory allocation.
url=https://github.com/bblanchon/ArduinoJson
architectures=*

I had to add the lines in red bellow to the file to get the library to work in the arduino 1.5 IDE:

name=ArduinoJson

version=4.0

author=Benoit Blanchon <http://blog.benoitblanchon.fr/>

maintainer=Benoit Blanchon <http://blog.benoitblanchon.fr/>

sentence=An efficient and elegant JSON library for Arduino

paragraph=Supports JSON parsing and formatting. Uses fixed memory allocation.

url=https://github.com/bblanchon/ArduinoJson

architectures=*

core-dependencies=

dependencies=

email=

After much thought I decided to leave the responsibility of parsing json to python scripts running on the linino side, so that arduino would just call the python script and read its stdout. I will be uploading my scripts to github at some point.

cloudClock: DS3231 RTC

Bought one of this from Dealextreme a while ago for this project, but only now I had the time to look into trying it. The exact one I bought is the one in the picture above, link here.

In order to use it, I searched for available libraries and turns out someone had already done all the work. The library can be found here and requires installing also the Time library, available here.  To connect it to the arduino is plain easy, vcc, ground, SDA and SCL and that's all. Sounded easy... plugged it, compiled the example and... FAIL... hmmm... it did not work... nothing... After reviewing the code and the soldering, could not see anything wrong, so I went to the basics... Arduino Yun pinout. If you see the Arduino Yun pinout you may notice how SDA and SCL are in 2 places and if you see my wiring diagram for the arduino yun internet radio, you will noticed that my rotary encoder is plugged in the pins carrying the SDA and SCL signals... oops... unplug the rotary encoder and there I had it... RTC working. So this is how it looks like now (corresponding change will have to be done to the yun radio code to adapt to the rotary encoder pin change, also the RTC module does not correspond to the one I am using, but pins are named identically):

Reverse engineering SJCAM4000 WiFi... well... kind of...

I got a so called Action Camera. Not enought interest on the whole video recording thing to spend a fortune on a GoPro, so I went for the Chinese option and bought a SJCAM 4000 wifi. I will not get into the whole review and unboxing discussion... that is not the purpose of this blog, you have plenty of pages already discussing that... google that...

Anyway, so the camera comes with a wifi option. How does that work? the camera opens an access point, you connect with an iOS or android app and you can configure and control the camera, see live video or download recordings (videos or pictured). That is great feature, I think, it saves you from having to extract the microSD card from the camera, which makes things easier, but I prefer using my laptop for downloading the pictures and videos than my handset, but no ubuntu app... grrrr.... had to figure out the protocol used in wifi mode...

I connected my laptop to the camera access point. I had to figure out the ip address of the camera:

arp -a

? (192.168.1.254) at 18:83:bf:XX:XX:XX [ether] on wlan0

Then I tried connecting to that IP with telnet and ssh without luck. Lastly I tried opening the webpage in that IP: http://192.168.1.254

Bingo!!! It opened a web page that allowed browsing the microSD card contents and even upload files, but no sight of configuration options or live video feed. That was really all I was looking for, but hell... I want the whole enchilada.

I installed an app in my android phone that allows capturing tcpdumps. I used tPacketCapture, which does not require having a rooted phone. If your phone is rooted, you can just use tcpdump from busybox. I captured a connection with my android app while I was doing some changes in the settings to inspect it later in the PC with wireshark. So here is what I found:

1. The live video feed is just an RTSP connection, pretty standard and no encryption... cool... So connected the laptop back to the camera, opened VLC, Media->Open Network Stream and typed the URL I saw in my tcpdump pcap file: rtsp://192.168.1.254/sjcam.mov voila... live video from the camera
2. The pcap file also had some http get requests. I could distinguish 2 types of requests, those with a parameter and those with out it. I assume the ones with no parameter are requests for info and the ones with a parameter are request for changing a setting. See the examples bellow (red is from laptop to camera, blue is from camera to laptop)

with no parameter:

GET /?custom=1&cmd=3016 HTTP/1.1

accept: */*

User-Agent: Dalvik/1.6.0 (Linux; U; Android 4.4.2; -------------------)

Host: 192.168.1.254

Connection: Keep-Alive

Accept-Encoding: gzip

HTTP/1.1 200 OK

Server: eCos/1.0

Cache-Control:no-store, no-cache, must-revalidate

Pragma: no-cache

Accept-Ranges: bytes

Content-length: 97

Content-type: text/xml

Connection: close

<?xml version="1.0" encoding="UTF-8" ?>

<Function>

<Cmd>3016</Cmd>

<Status>1</Status>

</Function>

with no parameter:

GET /?custom=1&cmd=2002&par=2 HTTP/1.1

accept: */*

User-Agent: Dalvik/1.6.0 (Linux; U; Android 4.4.2; -------------------)

Host: 192.168.1.254

Connection: Keep-Alive

Accept-Encoding: gzip

HTTP/1.1 200 OK

Server: eCos/1.0

Cache-Control:no-store, no-cache, must-revalidate

Pragma: no-cache

Accept-Ranges: bytes

Content-length: 97

Content-type: text/xml

Connection: close

<?xml version="1.0" encoding="UTF-8" ?>

<Function>

<Cmd>2002</Cmd>

<Status>0</Status>

</Function>

I did not dig further in this, but assume that each of those cmd numbers corresponds to a camera setting, and the parameter (par) is the value to set, but still need more research to figure out cmd and settings. I hope to make a python library to control the camera at some point, but will need some time to reverse engineer all the commands and parameters.

The last thing I did on my first weekend with the camera was try to watch the videos in my laptop and mobile phone, well, turns out the format of the recorded videos is ".MOV" and both mobile and laptop had issues with that format, even VLC which seems to play everything had issues: video was stopping and audio was out of sync. I converted the ".MOV" file to ".mp4" using avconv:

avconv -i input_file.MOV -vcodec copy -acodec mp2 output_file.mp4

What this does is generate an mp4 file in which the video is unaltered (no reencoding done, the video from the mov, h.264 compressed, is extracted and muxed again in the mp4 file without alteration). I have never been too keen on recompressing the video if it is already good from the source, unless there is a real need for it (reduce the file size). The audio gets compressed in mpeg layer 2 (audio codec in the ".MOV" is lpcm, no compression). Mp4 file format does not seem to support lpcm audio and I don't mind that much compressing the audio.

cloudClock: Force an update of the time with ntpd

Whenever ntpd detects a huge different between the system time and the ntp server time, ntpd does not update the time. I should investigate further on why it behaves that way, but I think I know enough about ntpd already for now... so I investigated how to make ntpd to force the update and this is what I found:

To force a large NTP update, do the following:

Stop the NTPD service ("/etc/init.c/ntpd stop")
run ntpd -gnqd
Start the NTPD service again ("/etc/init.c/ntpd start")

cloudClock: ntpstat

In order to find out the status of the ntp daemon, there seem to be 2 possibilities: ntpq or ntpstat.
ntpq is by far the one that gives more info, really too much for my liking, I just want to find out if the clock is in sync or not!! so I went for ntpstat. As I mentioned in my previous post, there is no ntpstat in linino, so I had to compile it from scratch. This was easy. I got the ntpstat source code from here and compiled it doing:

../linino/trunk/staging_dir/toolchain-mips_r2_gcc-4.6-linaro_uClibc-0.9.33.2/bin/mips-openwrt-linux-gcc ntpstat.c -o ntpstat

cloudClock: time keeping

Part of the beauty of the cloudClock is the fact that it does not require having to set the time. For that I will use ntpd to keep linino on time. Turns out that linino already has ntpd running:

/usr/sbin/ntpd -n -p 0.openwrt.pool.ntp.org -p 1.openwrt.pool.ntp.org -p 2.openwrt.pool.ntp.org -p 3.open

First thing I thought was that I would like to change the ntp servers being used and select some servers near by. I also wanted to the change the timezone to my current timezone. Turns out both those things are configured in the same file: /etc/config/system

Original file was:

config system
option hostname 'Arduino'
option timezone 'UTC'
option timezone_desc 'Rest of the World (UTC)'

config timeserver 'ntp'
list server '0.openwrt.pool.ntp.org'
list server '1.openwrt.pool.ntp.org'
list server '2.openwrt.pool.ntp.org'
list server '3.openwrt.pool.ntp.org'
option enable_server '0'

And my modified file is now:

config system
option hostname 'Arduino'
option timezone 'CET-1CEST,M3.5.0,M10.5.0/3'
option timezone_desc 'Europe/Madrid'

config timeserver 'ntp'
list server '0.pool.ntp.org'
list server '1.pool.ntp.org'
list server '2.pool.ntp.org'
list server '3.pool.ntp.org'
option enable_server '0'

I got the timezone to set from this list: http://wiki.openwrt.org/doc/uci/system#time.zones

I need a way to find out if ntpd is working correctly. There seems to be 2 possibilities: ntpstat and ntpq, but I could not find ntpstat in linino, so tried with ntpq. Unfortunately ntpq was not working either. My conclusion was that it was using busybox ntpd, rather than a fully featured ntpd, so I installed ntpd doing:

opkg update
opkg install ntpd
/etc/init.d/sysntpd disable
/etc/init.d/ntpd enable
/etc/init.d/ntpd start

Now ntpq works!!! just need to figure out how do I parse the output of ntpq to get something meaningful.

On the arduino side, I bought an RTC chip module to make sure the clock is on time, even when there is no internet connection. I will get back to that whenever the RTC module arrives.