Processing III: Merging multimodal data

Overview

In the previous scripts, we have preprocessed various motion and acoustic data. In this script, we will merge the data into a single file per trial. These data include:

Balance Board data
Kinematics
Joint angles
Joint moments
Amplitude envelope
f0
Formants
Spectral centroid

Code to prepare environment

# packages
import os
import glob
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import scipy
import matplotlib.pyplot as plt
import seaborn as sns

curfolder = os.getcwd()
print('Current folder is:', curfolder)

# folders with processed data
MTfolder_processed = os.path.join(curfolder, 'TS_motiontracking')
ACfolder_processed = os.path.join(curfolder, 'TS_acoustics')
# folder to save merged data
TSmerged = os.path.join(curfolder, 'TS_merged')
if not os.path.exists(TSmerged):
    os.makedirs(TSmerged)

# prepare all files
bbfiles = glob.glob(os.path.join(MTfolder_processed, 'bb*.csv'))
idfiles = glob.glob(os.path.join(MTfolder_processed, 'id*.csv'))
ikfiles = glob.glob(os.path.join(MTfolder_processed, 'ik*.csv'))
mtfiles = glob.glob(os.path.join(MTfolder_processed, 'mt*.csv'))
envfiles = glob.glob(os.path.join(ACfolder_processed, 'env*_norm.csv'))
f0files = glob.glob(os.path.join(ACfolder_processed, 'f0*.csv'))
formants = glob.glob(os.path.join(ACfolder_processed, '*formants*.csv'))
scfiles = glob.glob(os.path.join(ACfolder_processed, 'cog*.csv'))

if any(len(files)==0 for files in [bbfiles, idfiles, ikfiles, mtfiles, envfiles, f0files, formants, scfiles]):
    raise ValueError('One or more required data files are missing in the processed data folders.')

Current folder is: f:\FLESH_ContinuousBodilyEffort\03_TS_processing

When extracting and processing the acoustic and motion signals, we kept the sampling rates untouched. This means that now we have a variety of timeseries that each samples at different frequency. Inspecting a trial per each signal, we see the following sampling rates:

balance board: 500.00068202009317
inverse dynamics: 30.00120004800192
inverse kinematics: 60.00240009600385
kinematics: 60.00000000000024
envelope: 48000.00000000008
f0: 499.0910682304781
formants: 200000.00000000012
spectral centroid: 472.65436024146993

We opt for 500 Hz as the final sampling rate we will merge on. That means that we will interpolate all missing data (using linear interpolation) to match this frequency.

Additionally, we will adapt the formants such that we only consider values that are present within a range of an amplitude peak (see acoustics processing script for more details), or where f0 is present, or both. These two situations can be considered as yielding in the most reliable formant values.

Finally, we will also use inverse kinematics and dynamics to calculate power (as joint moment × joint velocity) and smooth it with 1st-polynomial Savitzky-Golay filter with span of 560 ms.

Custom functions

# Function to create chunks of non-NaN values in a dataframe
def create_chunks(df, var):
    """Create chunks of non-NaN values in a dataframe.

    Args:
        df: Input dataframe
        var: Column name to analyze for non-NaN values

    Returns:
        tuple: (annotated dataframe, array of unique chunk identifiers)
    """

    df['chunk'] = None

    # annotate chunks of non-NaN values
    chunk = 0
    for index, row in df.iterrows():
        if np.isnan(row[var]):
            continue
        else:
            df.loc[index, 'chunk'] = chunk
            # if the next value is NaN or this is the last row, increase the chunk
            if index == len(df)-1:
                continue
            elif np.isnan(df.loc[index+1, var]):
                chunk += 1

    chunks = df['chunk'].unique()

    if len(chunks) > 1: # skip if chunks are empty (that means that there is no f0 trace)
        # ignore the first chunk (None)
        chunks = chunks[1:]

    return df, chunks

# Function to interpolate chunks of non-NaN values in a dataframe to maintain discontinuities in the signal
def interpolate_chunks(df, chunks, var):
    """Interpolate chunks of non-NaN values in a dataframe to maintain discontinuities in the signal.

    Args:
        df: Input dataframe
        chunks: Array of unique chunk identifiers
        var: Column name to interpolate

    Returns:
        DataFrame: Dataframe with interpolated values
    """
    # we ignore the None chunk above, so if there is some trace, None should not be within chunks
    if None not in chunks:
        for chunk in chunks:
            # get the first and last row of the chunk
            firstrow = df[df['chunk'] == chunk].index[0]
            lastrow = df[df['chunk'] == chunk].index[-1]
            # fill all inbetween with the chunk number
            df.loc[firstrow:lastrow, 'chunk'] = chunk
            # get the rows of the chunk
            chunkrows = df[df['chunk'] == chunk].copy()
            # interpolate
            chunkrows[var] = chunkrows[var].interpolate(method='linear', x = chunkrows['time'])
            # put the interpolated chunk back to the df
            df.loc[df['chunk'] == chunk, var] = chunkrows[var]

    # get rid of the chunk column
    df.drop('chunk', axis=1, inplace=True)

    return df

Merging signals on a common sampling rate

import warnings
warnings.filterwarnings("ignore")

desired_sr = 0.5    # this is the sr we are going to merge on (in Hz/sec)

error_log = []

mergedfiles = glob.glob(os.path.join(TSmerged, 'merged_*.csv'))

for file in bbfiles:

    bb_df = pd.read_csv(file)
    trialid = bb_df['TrialID'][0]
    print('working on ' + trialid)
    
    # Find ID file
    id_name = 'id_' + trialid
    id_file = [x for x in idfiles if id_name in x]
    try:
        id_df = pd.read_csv(id_file[0])
    except IndexError:
        print('IndexError: ' + trialid + ' not found for ID')
        errormessage = 'IndexError: ' + trialid + ' not found for ID'
        error_log.append(errormessage)
        continue
    
    # Find MT file
    mt_name = 'mt_' + trialid
    mt_file = [x for x in mtfiles if mt_name in x]
    try:
        mt_df = pd.read_csv(mt_file[0])
        # check if there are two 0 in Time, drop the first one
        if mt_df['Time'][0] == 0 and mt_df['Time'][1] == 0:
            mt_df = mt_df.drop(index=0).reset_index(drop=True)
    except IndexError:
        print('IndexError: ' + trialid + ' not found for MT')
        errormessage = 'IndexError: ' + trialid + ' not found for MT'
        error_log.append(errormessage)
        continue
    # rename Time to time
    mt_df.rename(columns={'Time': 'time'}, inplace=True)

    # Find ENV file
    env_name = 'env_' + trialid
    env_file = [x for x in envfiles if env_name in x]
    try:
        env_df = pd.read_csv(env_file[0])
    except IndexError:
        print('IndexError: ' + trialid + ' not found for ENV')
        errormessage = 'IndexError: ' + trialid + ' not found for ENV'
        error_log.append(errormessage)
        continue
    # rename trialID to TrialID
    env_df.rename(columns={'trialID': 'TrialID'}, inplace=True)

    # Find F0 file
    f0_name = 'f0_' + trialid
    f0_file = [x for x in f0files if trialid in x]
    try:
        f0_df = pd.read_csv(f0_file[0])
    except IndexError:
        print('IndexError: ' + trialid + ' not found for F0')
        errormessage = 'IndexError: ' + trialid + ' not found for F0'
        error_log.append(errormessage)
        continue
    # rename time_ms to time
    f0_df.rename(columns={'time_ms': 'time'}, inplace=True)
    # rename ID to TrialID
    f0_df.rename(columns={'ID': 'TrialID'}, inplace=True)

    # Find IK file
    ik_name = 'ik_' + trialid   
    ik_file = [x for x in ikfiles if ik_name in x]
    try:
        ik_df = pd.read_csv(ik_file[0])
    except IndexError:
        print('IndexError: ' + trialid + ' not found for IK')
        errormessage = 'IndexError: ' + trialid + ' not found for IK'
        error_log.append(errormessage)
        continue

    # Find formant file
    # take first two element
    if 'rein' in trialid:
        formantid = trialid.split('_')[0] + '_' + trialid.split('_')[1] + '_rein_trial_' + trialid.split('_')[3] + '_'
    else:
        formantid = trialid.split('_')[0] + '_' + trialid.split('_')[1] + '_trial_' + trialid.split('_')[2] + '_'
    formants_file = [x for x in formants if formantid in x]
    try:
        formants_df = pd.read_csv(formants_file[0])
    except IndexError:
        print('IndexError: ' + trialid + ' not found for formants')
        errormessage = 'IndexError: ' + trialid + ' not found for formants'
        error_log.append(errormessage)
        continue
    # rename triald to TrialID
    formants_df['TrialID'] = trialid
    formants_df = formants_df[['time', 'f1', 'f2', 'f3', 'TrialID']]
    formants_df['time'] = formants_df['time'] * 1000

    # Find COG file
    if 'rein' in trialid:
        cogid = trialid.split('_')[0] + '_' + trialid.split('_')[1] + '_' + trialid.split('_')[3] + '_'
        cogname = 'cog_' + cogid
        sc_file = [x for x in scfiles if cogname in x]
    else:
        cogname = 'cog_' + trialid
        sc_file = [x for x in scfiles if cogname in x]
    try:
        sc_df = pd.read_csv(sc_file[0])
    except IndexError:
        print('IndexError: ' + trialid + 'not found CoG')
        errormessage = 'IndexError: ' + trialid + ' not found for CoG'
        error_log.append(errormessage)
        continue

    # write error log
    with open(os.path.join(TSmerged, 'error_log.txt'), 'w') as f:
        for item in error_log:
            f.write("%s\n" % item)

    ############## MERGING ########################

    #### regularize sr in bb
    time_new = np.arange(0, max(bb_df['time']), 1/desired_sr)
    bb_interp = pd.DataFrame({'time': time_new})
    
    # interpolate all columns in samplebb 
    colstoint = bb_df.columns
    colstoint = [x for x in colstoint if 'time' not in x]
    colstoint = [x for x in colstoint if 'TrialID' not in x]
    colstoint = [x for x in colstoint if 'FileInfo' not in x]

    for col in colstoint:
        bb_interp[col] = bb_df[col].interpolate(method='linear', x = bb_interp['time'])

    # add trialid and time
    bb_interp['TrialID'] = trialid
    bb_interp['FileInfo'] = bb_df['FileInfo'][0]

    ########### merge the bb_interp with env

    # merge the two dataframes
    merge1 = pd.merge(bb_interp, env_df, on=['time', 'TrialID'], how='outer')

    # interpolate missing values of envelope and audio
    colstoint = merge1.columns
    colstoint = [x for x in colstoint if 'audio' in x or 'envelope' in x]

    for col in colstoint: 
        merge1[col] = merge1[col].interpolate(method='linear', x = merge1['time'])

    # now we can kick out all values where COPc is NaN
    merge1 = merge1[~np.isnan(merge1['COPc'])]

    ########### merge with ID

    # merge the two dataframes
    merge2 = pd.merge(merge1, id_df, on=['time', 'TrialID'], how='outer')
    #merge2 = merge2.sort_values(by='time').reset_index(drop=True)

    # get cols of sampleid
    colstoint = id_df.columns
    colstoint = [x for x in colstoint if 'time' not in x]
    colstoint = [x for x in colstoint if 'TrialID' not in x]

    # interpolate 
    for col in colstoint:
        merge2[col] = merge2[col].interpolate(method='linear', x = merge2['time'])

    # now we can kick out all values where COPc is NaN to get sampling rate back to 500 
    merge2 = merge2[~np.isnan(merge2['COPc'])]

    ########### merge with MT
    # merge the two dataframes
    merge3 = pd.merge(merge2, mt_df, on=['time', 'TrialID'], how='outer')
    #merge3 = merge3.sort_values(by='time').reset_index(drop=True)

    # get cols of samplemt
    colstoint = mt_df.columns
    colstoint = [x for x in colstoint if 'time' not in x]
    colstoint = [x for x in colstoint if 'TrialID' not in x]

    # interpolate missing values of from mt
    for col in colstoint:
        merge3[col] = merge3[col].interpolate(method='linear', x = merge3['time'])

    # now we can kick out all values where COPc is NaN
    merge3 = merge3[~np.isnan(merge3['COPc'])]

    ########### merge with F0

    # for interpolation, we need to again parse f0 into chunks of non-NaN values
    f0_df, chunks = create_chunks(f0_df, 'f0')
    
    # now we can merge
    merge4 = pd.merge(merge3, f0_df, on=['time', 'TrialID'], how='outer')

    # interpolate f0 signal, while maintaining discontinuities
    merge4 = interpolate_chunks(merge4, chunks, 'f0')

    # now we can drop all rows where COPc is NaN
    merge4 = merge4[~np.isnan(merge4['COPc'])]

    ########### merge with IK

    merge5 = pd.merge(merge4, ik_df, on=['time', 'TrialID'], how='outer')

    # get cols of sampleik
    colstoint = ik_df.columns
    colstoint = [x for x in colstoint if 'time' not in x]
    colstoint = [x for x in colstoint if 'TrialID' not in x]

    # interpolate missing values of from ik
    for col in colstoint:
        merge5[col] = merge5[col].interpolate(method='linear', x = merge5['time'])

    # now we can kick out all values where COPc is NaN
    merge5 = merge5[~np.isnan(merge5['COPc'])]

    ########### merge with formants
    merge6 = pd.merge(merge5, formants_df, on=['time', 'TrialID'], how='outer')

    # get cols of sampleformants
    colstoint = formants_df.columns
    colstoint = [x for x in colstoint if 'time' not in x]
    colstoint = [x for x in colstoint if 'TrialID' not in x]

    # interpolate missing values of from formants - currently they do not have NaNs so we can interpolate the whole signal instead of only non-NaN chunks
    for col in colstoint:
        merge6[col] = merge6[col].interpolate(method='linear', x = merge6['time'])

    # now we can kick out all values where COPc is NaN
    merge6 = merge6[~np.isnan(merge6['COPc'])]

    ########### merge with CoG
    merge7 = pd.merge(merge6, sc_df, on=['time', 'TrialID'], how='outer')

    # get cols of samplespecCentroid
    colstoint = sc_df.columns
    colstoint = [x for x in colstoint if 'time' not in x]
    colstoint = [x for x in colstoint if 'TrialID' not in x]

    # for interpolation, we need to again parse specCentroid into chunks of non-NaN values
    sc, chunks = create_chunks(sc_df, 'CoG')
    
    # now we merge
    merge8 = pd.merge(merge7, sc, on=['time', 'TrialID', 'CoG'], how='outer')
    #merge8 = merge8.sort_values(by='time').reset_index(drop=True)

    # interpolate CoG signal, while maintaining discontinuities
    merge8 = interpolate_chunks(merge8, chunks, 'CoG')

    # now we can kick out all values where COPc is NaN
    merge8 = merge8[~np.isnan(merge8['COPc'])]

    # this is final df
    merge_final = merge8     

    ############## FORMANT ADAPTATION ########################

    # find peaks in envelope, with min=mean
    peaks, _ = scipy.signal.find_peaks(merge_final['envelope_norm'], height=np.mean(merge_final['envelope_norm']))
    # get widths of the peaks
    widths = scipy.signal.peak_widths(merge_final['envelope_norm'], peaks, rel_height=0.95)
    # peak width df with starts and ends
    peak_widths = pd.DataFrame({'start': widths[2], 'end': widths[3]})

    # now create a new column env_weak_width, and put 0s everywhere, and 1s in the intervals of the width
    merge_final['env_peak_width'] = 0
    for index, row in peak_widths.iterrows():
        merge_final.loc[int(row['start']):int(row['end']), 'env_peak_width'] = 1

    # now we will create formant columns, where we will keep only formants in the intervals of env_pak_width OR where f0 is not NaN
    merge_final['f1_clean_f0'] = merge_final['f1']
    merge_final['f2_clean_f0'] = merge_final['f2']
    merge_final['f3_clean_f0'] = merge_final['f3']

    # where f0 is NaN, we will put NaN - these are formants during f0 only
    merge_final.loc[np.isnan(merge_final['f0']), 'f1_clean_f0'] = np.nan
    merge_final.loc[np.isnan(merge_final['f0']), 'f2_clean_f0'] = np.nan
    merge_final.loc[np.isnan(merge_final['f0']), 'f3_clean_f0'] = np.nan

    # we will also create formants, where we will keep only those in the intervals of env_pak_width
    merge_final['f1_clean_env'] = merge_final['f1']
    merge_final['f2_clean_env'] = merge_final['f2']
    merge_final['f3_clean_env'] = merge_final['f3']

    # where env_peak_width is 0, we will put NaN - these are formants during envelope peaks only
    merge_final.loc[merge_final['env_peak_width'] == 0, 'f1_clean_env'] = np.nan
    merge_final.loc[merge_final['env_peak_width'] == 0, 'f2_clean_env'] = np.nan
    merge_final.loc[merge_final['env_peak_width'] == 0, 'f3_clean_env'] = np.nan

    ## now we create formants where we copy values from clean_env and clean_f0
    merge_final['f1_clean'] = merge_final['f1_clean_env']
    merge_final['f2_clean'] = merge_final['f2_clean_env']
    merge_final['f3_clean'] = merge_final['f3_clean_env']

    # where formant is now NaN, copy values from f_clean_f0 in case there is a value
    merge_final.loc[np.isnan(merge_final['f1_clean']), 'f1_clean'] = merge_final['f1_clean_f0']
    merge_final.loc[np.isnan(merge_final['f2_clean']), 'f2_clean'] = merge_final['f2_clean_f0']
    merge_final.loc[np.isnan(merge_final['f3_clean']), 'f3_clean'] = merge_final['f3_clean_f0']

    # now calculate formant velocities (but only for the f_clean)
    merge_final['f1_clean_vel'] = np.insert(np.diff(merge_final['f1_clean']), 0, 0)
    merge_final['f2_clean_vel'] = np.insert(np.diff(merge_final['f2_clean']), 0, 0)
    merge_final['f3_clean_vel'] = np.insert(np.diff(merge_final['f3_clean']), 0, 0)

    # smooth
    merge_final['f1_clean_vel'] = scipy.signal.savgol_filter(merge_final['f1_clean_vel'], 5, 3)
    merge_final['f2_clean_vel'] = scipy.signal.savgol_filter(merge_final['f2_clean_vel'], 5, 3)
    merge_final['f3_clean_vel'] = scipy.signal.savgol_filter(merge_final['f3_clean_vel'], 5, 3)

    # multiply by sr (500) to get formant velocity in Hz/s
    sr = 500
    merge_final['f1_clean_vel'] = merge_final['f1_clean_vel'] * sr
    merge_final['f2_clean_vel'] = merge_final['f2_clean_vel'] * sr
    merge_final['f3_clean_vel'] = merge_final['f3_clean_vel'] * sr

    ########## POWER ####################
    groups = ['lowerbody', 'leg', 'head', 'arm']

    for group in groups:
        # get all columns that contain group
        cols = [x for x in merge_final.columns if group in x]

        # get all columns that contain 'moment_sum'
        torque = [x for x in cols if 'moment_sum' in x]
        # but not change
        torque = [x for x in torque if 'change' not in x][0]

        # get all columns that contain 'angSpeed_sum'
        angSpeed = [x for x in cols if 'angSpeed_sum' in x][0]

        # get power which is moment * angSpeed
        merge_final[group + '_power'] = merge_final[torque] * merge_final[angSpeed]
        # smooth
        try:
            merge_final[group + '_power'] = scipy.signal.savgol_filter(merge_final[group + '_power'], 281, 1) # window 281 corresponds to 562 ms (we add +1 to have odd length of window)
        except ValueError:
            # if there is a ValueError, it means that the window is too large for the data, so we will use a smaller window
            try:
                merge_final[group + '_power'] = scipy.signal.savgol_filter(merge_final[group + '_power'], 101, 2) # window 101 corresponds to 202 ms (we add +1 to have odd length of window)
            except ValueError:
                merge_final[group + '_power'] = scipy.signal.savgol_filter(merge_final[group + '_power'], 51, 2)

    # write to csv
    merge_final.to_csv(os.path.join(TSmerged, 'merged_' + trialid + '.csv'), index=False)

Here is an example of the file containing all the data

	time	left_back	right_forward	right_back	left_forward	COPXc	COPYc	COPc	TrialID	FileInfo	...	f1_clean	f2_clean	f3_clean	f1_clean_vel	f2_clean_vel	f3_clean_vel	lowerbody_power	leg_power	head_power	arm_power
0	0.0	1.345838	1.104859	1.725188	1.559900	1.439532e-04	0.000883	0.447404	3_2_29_p1	p1_man_geluiden_c0	...	NaN	NaN	NaN	NaN	NaN	NaN	26.387555	0.531298	1.736917	42.046841
1	2.0	1.345150	1.105089	1.725022	1.560288	2.429427e-04	0.000944	0.487571	3_2_29_p1	p1_man_geluiden_c0	...	NaN	NaN	NaN	NaN	NaN	NaN	26.288702	0.529428	1.732889	41.879961
2	4.0	1.344578	1.105388	1.724908	1.560662	3.095784e-04	0.000986	0.517209	3_2_29_p1	p1_man_geluiden_c0	...	NaN	NaN	NaN	NaN	NaN	NaN	26.189848	0.527558	1.728861	41.713081
3	6.0	1.344116	1.105747	1.724842	1.561033	3.475877e-04	0.001013	0.535807	3_2_29_p1	p1_man_geluiden_c0	...	NaN	NaN	NaN	NaN	NaN	NaN	26.090995	0.525687	1.724833	41.546201
4	8.0	1.343758	1.106159	1.724818	1.561408	3.604821e-04	0.001026	0.544156	3_2_29_p1	p1_man_geluiden_c0	...	NaN	NaN	NaN	NaN	NaN	NaN	25.992141	0.523817	1.720805	41.379320
5	10.0	1.343499	1.106616	1.724831	1.561796	3.515632e-04	0.001028	0.543638	3_2_29_p1	p1_man_geluiden_c0	...	NaN	NaN	NaN	NaN	NaN	NaN	25.893288	0.521947	1.716777	41.212440
6	12.0	1.343334	1.107113	1.724876	1.562201	3.239276e-04	0.001021	0.535917	3_2_29_p1	p1_man_geluiden_c0	...	NaN	NaN	NaN	NaN	NaN	NaN	25.794434	0.520076	1.712749	41.045560
7	14.0	1.343255	1.107642	1.724949	1.562629	2.804723e-04	0.001007	0.522797	3_2_29_p1	p1_man_geluiden_c0	...	NaN	NaN	NaN	NaN	NaN	NaN	25.695580	0.518206	1.708721	40.878680
8	16.0	1.343257	1.108197	1.725044	1.563084	2.238998e-04	0.000987	0.506151	3_2_29_p1	p1_man_geluiden_c0	...	NaN	NaN	NaN	NaN	NaN	NaN	25.596727	0.516336	1.704693	40.711800
9	18.0	1.343334	1.108772	1.725158	1.563568	1.567239e-04	0.000962	0.487874	3_2_29_p1	p1_man_geluiden_c0	...	NaN	NaN	NaN	NaN	NaN	NaN	25.497873	0.514465	1.700665	40.544920
10	20.0	1.343479	1.109362	1.725286	1.564085	8.127456e-05	0.000936	0.469834	3_2_29_p1	p1_man_geluiden_c0	...	NaN	NaN	NaN	NaN	NaN	NaN	25.399020	0.512595	1.696637	40.378040
11	22.0	1.343687	1.109961	1.725424	1.564635	-2.966553e-07	0.000907	0.453796	3_2_29_p1	p1_man_geluiden_c0	...	NaN	NaN	NaN	NaN	NaN	NaN	25.300166	0.510725	1.692608	40.211160
12	24.0	1.343950	1.110564	1.725567	1.565217	-8.601123e-05	0.000878	0.441313	3_2_29_p1	p1_man_geluiden_c0	...	NaN	NaN	NaN	NaN	NaN	NaN	25.201313	0.508854	1.688580	40.044280
13	26.0	1.344262	1.111166	1.725713	1.565833	-1.740584e-04	0.000849	0.433575	3_2_29_p1	p1_man_geluiden_c0	...	NaN	NaN	NaN	NaN	NaN	NaN	25.102459	0.506984	1.684552	39.877400
14	28.0	1.344617	1.111763	1.725856	1.566481	-2.627900e-04	0.000821	0.431273	3_2_29_p1	p1_man_geluiden_c0	...	NaN	NaN	NaN	NaN	NaN	NaN	25.003606	0.505114	1.680524	39.710520
15	30.0	1.345009	1.112350	1.725995	1.567159	-3.507148e-04	0.000795	0.434517	3_2_29_p1	p1_man_geluiden_c0	...	NaN	NaN	NaN	NaN	NaN	NaN	24.904752	0.503243	1.676496	39.543640
16	32.0	1.345432	1.112924	1.726124	1.567864	-4.364936e-04	0.000770	0.442866	3_2_29_p1	p1_man_geluiden_c0	...	NaN	NaN	NaN	NaN	NaN	NaN	24.805899	0.501373	1.672468	39.376760
17	34.0	1.345878	1.113480	1.726242	1.568593	-5.189336e-04	0.000748	0.455457	3_2_29_p1	p1_man_geluiden_c0	...	NaN	NaN	NaN	NaN	NaN	NaN	24.707045	0.499503	1.668440	39.209880
18	36.0	1.346342	1.114015	1.726345	1.569344	-5.969836e-04	0.000728	0.471188	3_2_29_p1	p1_man_geluiden_c0	...	NaN	NaN	NaN	NaN	NaN	NaN	24.608191	0.497632	1.664412	39.043000
19	38.0	1.346817	1.114527	1.726430	1.570111	-6.697278e-04	0.000712	0.488889	3_2_29_p1	p1_man_geluiden_c0	...	NaN	NaN	NaN	NaN	NaN	NaN	24.509338	0.495762	1.660384	38.876120

20 rows × 529 columns

And here we visualize some of the timeseries. We can also see that our interpolation did maintain the original discontinuities in the f0 signal.

Now we are ready for extracting the features of interest