c# - How can I make this LINQ query of an Enumerable DataTable of GTFS data faster? -

-

i'm working gtfs data new york city mta subway system. need find stop times each route @ specific stop. that, stop times stoptimes datatable have, specific stop_id. want stop times between , next 2 hours.

then, need lookup trip each stop time, using trip_id value. trip, have lookup route, using route_id value, in order route name or number stop time.

here counts each datatable: stoptimes(522712), trips(19092), routes(27).

right now, takes anywhere 20 seconds 40 seconds execute. how can speed up? , suggestions appreciated. thanks!

foreach (var r in stoptimes.orderby(z => z.field<datetime>("departure_time").timeofday)                            .where(z => z.field<string>("stop_id") == stopid &&                                   z["departure_time"].tostring() != "" &&                                   z.field<datetime>("departure_time").timeofday >= datetime.utcnow.addhours(-5).timeofday &&                                   z.field<datetime>("departure_time").timeofday <= datetime.utcnow.addhours(-5).addhours(2).timeofday))         {             var trip = (from z in trips                         z.field<string>("trip_id") == r.field<string>("trip_id") &&                               z["route_id"].tostring() != ""                         select z).single();              var route = (from z in routes                          z.field<string>("route_id") == trip.field<string>("route_id")                          select z).single();              // stuff (not time-consuming)         }

try this:

var = datetime.utcnow; var tod0 = now.addhours(-5).timeofday; var tod1 = now.addhours(-5).addhours(2).timeofday;  var sts =     st in stoptimes     let stopid = st.field<string>("stop_id")     stopid == stopid     st["departure_time"].tostring() != ""     let departuretime = st.field<datetime>("departure_time").timeofday     departuretime >= tod0     departuretime >= tod1     let tripid = st.field<string>("trip_id")     select new     {         stopid,         tripid,         departuretime,     };

note there no orderby in query , we're returning anonymous type. "do stuff (not time-consuming)" code run may need add more properties.

the same approach happens trips & routes.

var ts =     t in trips     t["route_id"].tostring() != ""     let tripid = t.field<string>("trip_id")     let routeid = t.field<string>("route_id")     select new     {         tripid,         routeid,     };  var rs =     r in routes     let routeid = r.field<string>("route_id")     select new     {         routeid,     };

since you're getting single record each using todictionary(...) choice use.

var triplookup = ts.todictionary(t => t.tripid); var routelookup = rs.todictionary(r => r.routeid);

now query looks this:

var query = stoptime in sts.toarray()             let trip = triplookup[stoptime.tripid]             let route = routelookup[trip.routeid]             orderby stoptime.departuretime             select new             {                 stoptime,                 trip,                 route,             };

notice i've used .toarray() , i've put orderby right @ end.

and run code this:

foreach (var q in query) {     // stuff (not time-consuming) }

let me know if helps.


Comments

Post a Comment

Popular posts from this blog

c# - How to set Z index when using WPF DrawingContext? -

-
how set z-index drawing object when using drawingcontext.drawxxx() methods? the object drawn last have higher z index. can't change index of drawn objects. way draw in order. if using wpf (as placed tag), can use, example, canvas control. create shapes need like polyline obj = new polyline(); //... // ... set properties of obj and add them canvas uielementcollection: yourcanvasname.children.add(obj); //or yourcanvasname.children.insert(i, obj); first items of collection have higher z index. advantages in way: no need redraw on window changes, can anytime move objects , change order.
Read more

razor - Is this a bug in WebMatrix PageData? -

-
i think may have found bug in webmatrix's pagedata, not sure. concerns how pass data partial page calling page. in webmatrix documentation (tutorials, e.g. " 3 - creating consistent look ", , example code), pagedata recommended mechanism pass data between pages (e.g. content page layout page, or partial page). however have found not work other way, pass data partial page calling page. modifying or adding entries in pagedata in partial page, not seem calling page. cutting right down simplest possible example, in test page may have this: @{ pagedata["test"] = "initial entry"; } <p>before calling partial page, test value @pagedata["test"]</p> @renderpage("_testpartial.cshtml") <p>after returning calling page, test value @pagedata["test"]</p> and in _testpartial.cshtml page might have this: @{ pagedata["test"] = "modified entry"; } <p>in partial page...
Read more

android - layout with fragment and framelayout replaced by another fragment and framelayout -

-
edit: so after comments below, revisted , realized hanging me up. imagine client list , client details activity started : public class clientsmainactivity extends fragmentactivity { @override public void oncreate(bundle savedinstancestate) { super.oncreate(savedinstancestate); //studiotabopenhelper db; setcontentview(r.layout.main_client_activity); } } so works great, starts main_client_activity (defined in layout below, , call activity when button on main screen clicked): intent intent = new intent(getactivity(), clientsmainactivity.class); startactivity(intent); easy issue is, clientsmainactivity not call oncreateview or anything, sets layout layout defines fragment, , listfragment. fine cause not trying pass clientsmainactivity , if have hypothetical activity like: sessionmainsactivity called when click on session edit of client, not calling sessionsmainactivity same way (starts activity sets alayout), want layout set defines...
Read more