python 2.7 - Round pandas datetime index? -

-

i reading multiple spreadsheets of timeseries pandas dataframe , concatenating them common pandas datetime index. datalogger logged timeseries not 100% accurate makes resampling annoying because depending on if time higher or lower interval being sampled create nans , starts make series broken line. here's code 

def loaddata(filepaths):     t1 = time.clock()     in range(len(filepaths)):         xl = pd.excelfile(filepaths[i])         df = xl.parse(xl.sheet_names[0], header=0, index_col=2, skiprows=[0,2,3,4], parse_dates=true)         df = df.dropna(axis=1, how='all')          df = df.drop(['decimal year day', 'decimal year day.1', 'record'], axis=1)          if == 0:             dfs = df         else:             dfs = concat([dfs, df], axis=1)     t2 = time.clock()     print "files loaded dataframe in %s seconds" %(t2-t1)  files = ["london lysimeters corrected 5min.xlsx", "london water balance 5min.xlsx"] data = loaddata(files)

here's idea of index:

data.index

class 'pandas.tseries.index.datetimeindex'> [2012-08-27 12:05:00.000002, ..., 2013-07-12 15:10:00.000004] length: 91910, freq: none, timezone: none

what fastest , general round index nearest minute?

here's little trick. datetimes in nanoseconds (when viewed np.int64). round minutes in nanoseconds.

in [75]: index = pd.datetimeindex([ timestamp('20120827 12:05:00.002'), timestamp('20130101 12:05:01'), timestamp('20130712 15:10:00'), timestamp('20130712 15:10:00.000004') ])  in [79]: index.values out[79]:  array(['2012-08-27t08:05:00.002000000-0400',        '2013-01-01t07:05:01.000000000-0500',        '2013-07-12t11:10:00.000000000-0400',        '2013-07-12t11:10:00.000004000-0400'], dtype='datetime64[ns]')  in [78]: pd.datetimeindex(((index.asi8/(1e9*60)).round()*1e9*60).astype(np.int64)).values out[78]:  array(['2012-08-27t08:05:00.000000000-0400',        '2013-01-01t07:05:00.000000000-0500',        '2013-07-12t11:10:00.000000000-0400',        '2013-07-12t11:10:00.000000000-0400'], dtype='datetime64[ns]')

Comments

Post a Comment

Popular posts from this blog

html5 - What is breaking my page when printing? -

-
Image
i have questioner , don't know why when printing it's creating break before first question. live demo (link) i able resolve issue removing fieldset container had wrapped in.. having fieldset element causing of fields grouped together, , since not fit on first page, being pushed second page. removing fieldset element, able firefox print first 2 questions on page 1 identically chrome. it seems not new problem firefox , fieldset not playing nice found old bugzilla ticket 2008 has persisted , still being commented on in 2014 (link) if retaining fieldset important, found solution else came here (link) <script type='text/javascript'> $(window).bind('beforeprint', function(){ $('fieldset').each( function(item) { $(this).replacewith($('<div class="fieldset">' + this.innerhtml + '</div>')); } ) }); $(window).bind(
Read more

c# - must be a non-abstract type with a public parameterless constructor in redis -

-
when save object, got below error must non-abstract type public parameterless constructor in order use parameter 't' in generic type or method 'servicestack.redis.redisclient.store<t>(t) redisclass.getinstnace().store(msg); // error here redisclass.getinstnace().save(); as third party's class, can not edit how save object? could create wrapper around third party object call constructor, store wrapper? e.g. public class mywrapper { public thirdpartyobject thirdpartyinstance { get; set; } public mywrapper() { thirdpartyinstance = new thirdpartyobject("constructors"); } }
Read more

ajax - PHP/JSON Login script (Twitter style) not setting sessions -

-
i'm having trouble getting twitter style login work properly, reason gets stuck at session_register("useremail"); session_register("userpass"); and doesn't redirect after it. if enter wrong account data, works , appropriate error message pops up. this full code: <?php if(empty($_post['email']) || empty($_post['password'])) { die(msg(0,"all fields required")); } if(!(preg_match("/^[\.a-z0-9_\-\+]+[@][a-z0-9_\-]+([.][a-z0-9_\-]+)+[a-z]{1,4}$/", $_post['email']))) die(msg(0,"you haven't provided valid e-mail")); $host="localhost"; $username="root"; $password="x"; $db_name="x"; $tbl_name="x"; mysql_connect("$host", "$username", "$password")or die("cannot connect"); mysql_select_db("$db_name")or die("cannot select db"); $useremail=$_post['email']; $userpass=
Read more