Integrating iPod/Nike workout data into Fitbit

Recently, I received (and subsequently lost) a Fitbit Force for Christmas. The unfortunate discrepancies in the hardware aside, I enjoyed the unit and the wealth of data it collected about my daily activities. Fitbit was also gracious enough to allow developer access to all that collected information through a well-documented API.

While at the gym, I use a Polaris WearLink+ heart rate monitor along with an iPod Nano to gauge my cardiovascular workouts. The heart rate monitor chest strap detects my heart rate and transmits the information to the iPod Nano where it is recorded during my workout. This information is then sent to the Nike website when you sync the iPod. Even though the Nike website is sufficient enough to get a handle on the results of my workout, collating the data from both the Nike and Fitbit sites would be tedious at best.

After a bit of browsing through the mounted volume of the iPod Nano, I came across the following folder:

/iPod_Control/Device/Trainer/Workouts/Empeds

 which contained a subfolder titled 'heartrateonly' with files named with the date and time of my recent workouts in XML format.

Time to dust off the old Perl interpreter.

Implementation

<?xml version="1.0" encoding="UTF-8"?>
<sportsData type="heartRate">
   <vers>10</vers>
   <runSummary>
      <workoutName>Heart Rate Workout</workoutName>
      <time>2014-01-23T17:10:09-05:00</time>
      <duration>1501182</duration>
      <durationString>25'01"</durationString>
      <calories>0</calories>
      <heartRate>
         <average>122</average>
         <minimum>
            <duration>340552</duration>
            <bpm>112</bpm>
         </minimum>
         <maximum>
            <duration>295531</duration>
            <bpm>160</bpm>
         </maximum>
         <battery>3</battery>
      </heartRate>
      <musicChoice>2</musicChoice>
      <playlistList>
         <playlist>
            <playlistName>Electro House</playlistName>
         </playlist>
      </playlistList>
   </runSummary>
   <template>
      <templateID>48524F00</templateID>
      <templateName>Heart Rate Workout</templateName>
   </template>
   <activity>
      <type>bicycle</type>
      <name>Bicycling</name>
   </activity>
   <userInfo>
      <weight>83.0</weight>
      <device>iPod</device>
      <hrsID></hrsID>
   </userInfo>
   <ipodInfo>
      <model>MC525LL</model>
      <softwareVersion>1.2 (36B10147)</softwareVersion>
      <serialNumber></serialNumber>
   </ipodInfo>
   <startTime>2014-01-23T17:10:09-05:00</startTime>
   <endTime>2014-01-23T17:35:25-05:00</endTime>
   <snapShotList snapShotType="userClick">
      <snapShot event="pause">
         <duration>1394311</duration>
         <bpm>141</bpm>
      </snapShot>
      <snapShot event="resume">
         <duration>1394311</duration>
         <bpm>141</bpm>
      </snapShot>
      <snapShot event="pause">
         <duration>1497876</duration>
         <bpm>145</bpm>
      </snapShot>
      <snapShot event="resume">
         <duration>1497876</duration>
         <bpm>143</bpm>
      </snapShot>
      <snapShot event="pause">
         <duration>1501182</duration>
         <bpm>141</bpm>
      </snapShot>
      <snapShot event="onDemandVP">
         <duration>1501182</duration>
         <bpm>136</bpm>
      </snapShot>
      <snapShot event="stop">
         <duration>1501182</duration>
         <bpm>132</bpm>
      </snapShot>
   </snapShotList>
   <extendedDataList>
      <extendedData dataType="heartRate" intervalType="time" intervalUnit="s" intervalValue="10">108, 106, 87, 68, 62, 60, 62, 62, 0, 112, 111, 111, 110, 118, 122, 67, 120, 124, 122, 113, 113, 63, 63, 141, 37, 51, 68, 49, 39, 160, 32, 33, 0, 113, 112, 114, 116, 116, 115, 114, 114, 114, 115, 118, 118, 120, 121, 120, 118, 117, 118, 120, 120, 120, 119, 120, 121, 121, 121, 122, 122, 120, 120, 121, 120, 121, 123, 123, 123, 123, 122, 120, 120, 120, 122, 122, 123, 124, 123, 125, 124, 123, 123, 124, 124, 125, 125, 124, 124, 125, 126, 127, 127, 127, 127, 126, 126, 127, 127, 128, 129, 128, 129, 130, 131, 132, 132, 130, 130, 130, 130, 129, 128, 127, 129, 129, 130, 130, 131, 131, 130, 130, 130, 131, 130, 129, 130, 130, 130, 131, 132, 131, 132, 134, 135, 137, 140, 142, 141, 141, 142, 145, 148, 151, 152, 151, 151, 150, 148, 146</extendedData>
   </extendedDataList>

The XML file is, understandably, rather straight forward. As you can see from the code above, all the relevant pieces (and much more) are available to reconstruct my workout. For integration into the Fitbit, however, all we need are contained in these two tags:

<time>, which is the date and time the recording began and
<extendedData>, which contains the heart rate snapshots.

Extracting that information is easy. I use the XML::Twig module to parse the XML file and pull out the relative data. The heart rate data is stored as a comma-separated list of values that represent the BPM recorded at the intervals stored in the intervalType, intervalUnit and intervalValue attributes of the extendData tag.

Next, I needed to get the data into my Fitbit profile. Fitbit uses OAuth to allow applications to to interact with your profile. This way, a user can grant and manage which applications can have access and contribute to your data. Since we are the owners of the application and of the data, there are two steps involved in this process. First, we have to register a new application through the Fitbit website (as outlined in the documentation) which will supply us with an API consumer key and secret. These are the two pieces of information that lets Fitbit know what application is going to be interacting with them. We then have to run a small script and visit the Fitbit website in order to obtain our authentication token and token secret for our primary script.

# This is used to obtain the fitbit OAuth credentials

require LWP::Authen::OAuth;
use Net::OAuth::Client;
use strict;


# These have to be supplied from your fitbit registered applications

my $consumer_secret="";
my $consumer_key="";


# New OAuth object
my $ua = LWP::Authen::OAuth->new(
        oauth_consumer_secret => $consumer_secret,
);
 
# request a 'request' token
my $r = $ua->post( "http://api.fitbit.com/oauth/request_token",
        [
                oauth_consumer_key => $consumer_key,
        ]
);
die $r->as_string if $r->is_error;
 
# update the token secret from the HTTP response
$ua->oauth_update_from_response( $r );
 
# extract OAuth token
my $uri = URI->new( 'http:' );
$uri->query( $r->content );
my %oauth_data = $uri->query_form;
 
 
print "Visit the following address in your browser and approve the request:nnhttp://www.fitbit.com/oauth/authorize?oauth_token=$oauth_data{oauth_token}nnThen, enter supplied PIN: ";
my $oauth_verifier=<STDIN>;
chop($oauth_verifier);
 
# Request OAuth access token, secret
$r = $ua->post( "http://api.fitbit.com/oauth/access_token", [
        oauth_consumer_key => $consumer_key,
        oauth_verifier => $oauth_verifier,
]);
die $r->as_string if $r->is_error;

# Extract and display OAuth token and OAuth Token Secret
$uri = URI->new( 'http:' );
$uri->query( $r->content );
my %oauth_token_data = $uri->query_form;

print "OAuth token: ".$oauth_token_data{oauth_token}."n";
print "OAuth token secret: ".$oauth_token_data{oauth_token_secret}."n";

There is not much to see here. Most of the heavy lifting is being done by the LWP::Authen::OAuth and  Net::OAuth::Client modules to interact with Fitbit's server - we just need to edit the script and supply the consumer key and secret we received when we registered the application and then follow the prompts. Executing the script does the following:

• Requests a 'request' token in order to allow our application to register without providing our Fitbit authentication credentials (username and password) directly to it
• Generates a URL to visit in a web browser and obtain the token (PIN)
• Prompts the user to enter the PIN
• Registers the application

This is almost everything we need to record the workouts within the Fitbit tracking system. In order to create a manual log entry, we need to have the activity start time, duration and a calorie count. Using the heart rate data from the iPod, the calorie count can be tallied using the following formula (found on shapesense.com):

So, calculating the total calorie count is just a matter of iterating through all the recorded heart rate data points and applying this formula. Below is the complete code.

use XML::Twig;
use Date::Parse;
use Date::Format;
require LWP::Authen::OAuth;
use Net::OAuth::Client;

use strict;

#
# This information is from the fitbit application registration
# and from the script used to obtain the credentials

my $consumer_secret="";
my $consumer_key="";
my $oauth_token="";
my $oauth_token_secret="";

# the path to the XML file on the iPod Nano mounted volume

my $filepath="/Volumes/iPod Nano/iPod_Control/Device/Trainer/Workouts/Empeds/heartrateonly/latest/________.xml";

# personal information about the heart rate sensor user used to calculate caloric output

my $hr_user_vo2max=15;
my $hr_user_weight=175/2.2046;  # This is in LBS. The division operation is for metric conversion
my $hr_user_age=42;


# create the twig and parse the XML

my $twig=XML::Twig->new( pretty_print => 'indented');    # create the twig
$twig->parsefile($filepath); 

my $starttime=$twig->get_xpath('startTime',0)->text;
my $endtime=$twig->get_xpath('endTime',0)->text;
my $activity=$twig->get_xpath('activity/name',0)->text;
my $duration=$twig->get_xpath('runSummary/duration',0)->text;

my ($start_sec,$start_min,$start_hour,$start_mday,$start_mon,$start_year,$start_wday,$start_yday,$start_isdst) = localtime(str2time($starttime));

print "Activity     : ".$activity."n";
print "Workout Start: ".time2str("%a %b %o, %Y %l:%M%p%n",str2time($starttime));
print "Duration     : ".int((str2time($endtime)-str2time($starttime))/60)."m".((str2time($endtime)-str2time($starttime))-(int((str2time($endtime)-str2time($starttime))/60)*60))."snn";


# find all the events

my %events;
foreach my $x ($twig->get_xpath('snapShotList/',0)->children) {

	$events{$x->get_xpath('duration',0)->text}=$x->{'att'}->{'event'};
	
}

# merge the heart rate data with the events

foreach my $x ($twig->get_xpath('extendedDataList/',0)->children) {

	if ($x->{'att'}->{'dataType'} eq 'heartRate') {
	
		my @hr=split(/, /, $x->text);
		my $intv=$x->{'att'}->{'intervalValue'};
		my $i=0;
		foreach (@hr) {
		
			$events{($i*$intv*1000)} = "HR: ".$_.",$intv";
			$i++;
			
		}

	}
	
}

my $total_calories=0;
my $interim_calories=0;
my $last_event=0;

#
# Display the heart rate between events
# this is for future expansion into a system that would allow me to separate my workouts between machines
# and have them automatically added and ignore cooldowns/warmups/etc.

foreach (sort {$a<=>$b} keys %events) {

	#print $_.":".$events{$_};
	
	if ($last_event==0) {
		$last_event=$_;
	}
	
	if ($events{$_}=~/HR: (d*),(d*)/) {
	
		my $heart_rate=$1;
		my $heart_rate_duration=(1/3600)*$2;
		my $calories=((-95.7735 + (0.634 * $heart_rate) + (0.404 * $hr_user_vo2max) + (0.394 * $hr_user_weight) + (0.271 * $hr_user_age))/4.184) * 60 * $heart_rate_duration;
		$total_calories+=$calories;
		$interim_calories+=$calories;
	
	} else {
	
		if ($interim_calories>0) {
		
			my ($start_sec,$start_min,$start_hour,$start_mday,$start_mon,$start_year,$start_wday,$start_yday,$start_isdst) = localtime(str2time($starttime)+($_/1000));
		
			print "t@ ".$start_hour.":".$start_min.":  Elapsed time since last event: ".int((($_-$last_event)/1000)/60)."M".int(((($_-$last_event)/1000))-(int((($_-$last_event)/1000)/60)*60))."S, ".$interim_calories." cal. burnedn";
			$interim_calories=0;
			$last_event=$_;
			
		}
	
	}
	
}

print "Total Calories Burned: ".$total_calories."nn";

#
# Log the information to Fitbit

my $ua = LWP::Authen::OAuth->new(
        oauth_consumer_key => $consumer_key,
        oauth_consumer_secret => $consumer_secret,
        oauth_token => $oauth_token,
        oauth_token_secret => $oauth_token_secret,
);
 
my $r=$ua->post( 'http://api.fitbit.com/1/user/-/activities.json', [

	activityName =>		"iPod HR: ".$activity,
	manualCalories =>	int($total_calories+0.5),
	startTime =>		sprintf("%02d:%02d",$start_hour,$start_min),
	durationMillis =>	$duration,
	date => 			sprintf("%04d-%02d-%02d",$start_year+1900, $start_mon+1, $start_mday),

]);
die $r->as_string if $r->is_error;

print "Fitbit profile updated.nn";

Since manually running this script each time is barely more convenient that entering the information manually, future plans for this script include processing the files automatically when the iPod Nano is attached to the USB port of my computer and to store all the heart rate information into a database. The XML file also records timestamps each time the power button is pressed on the iPod Nano during the workout (which gives an auditory prompt with current workout information) which can be used to segment my workout data to make it easier to move between equipment at the gym and to gauge recovery times.