Travel Mapping

User Discussions => Welcome & Notices => Topic started by: Jim on January 22, 2017, 10:43:58 pm

Title: Temporary Suspension of Graph Generation
Post by: Jim on January 22, 2017, 10:43:58 pm
Over the next 9-10 days, my algorithms students will be working with TM-derived graph data for a lab assignment.  Since the graphs files change every time I regenerate them even if the underlying data does not (waypoint order, etc), I plan to stop the regeneration of new graphs until this lab is complete so I and my students will have consistent graphs to work with.  Apologies to those who are loading them up and fighting the good fight against NMPs, bad concurrencies, and other issues.  I should be able to restart the generation of new graphs on each site update around February 1 or February 2.

Tonight's site update did include new graphs, so they are currently consistent with what's in GitHub and in the DB.  That will no longer be the case after the next site update.
Title: Re: Temporary Suspension of Graph Generation
Post by: froggie on January 23, 2017, 07:20:28 am
Would halting site updates for the next week and a half accomplish the same purpose?  Seems it'd be simpler to just do that...
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on January 23, 2017, 10:06:52 am
Would halting site updates for the next week and a half accomplish the same purpose?  Seems it'd be simpler to just do that...

No, it's simple enough to perform all except the graph generation.  I add a "-k" flag to one script and it skips those steps.
Title: Re: Temporary Suspension of Graph Generation
Post by: si404 on January 23, 2017, 11:12:15 am
Would halting site updates for the next week and a half accomplish the same purpose?  Seems it'd be simpler to just do that...

No, it's simple enough to perform all except the graph generation.  I add a "-k" flag to one script and it skips those steps.
With added bonus of not penalising the website bunch much.
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on January 23, 2017, 11:48:53 am
A better future solution is likely a mechanism where I'd archive "release" versions of the graphs periodically, and have students stick with a particular release version for an assignment or maybe a whole semester.  But that didn't happen in time.
Title: Re: Temporary Suspension of Graph Generation
Post by: sipes23 on January 24, 2017, 12:37:07 am
I'm excited that this site has academic applications.
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on January 24, 2017, 07:33:07 am
I'm excited that this site has academic applications.

It's how I help justify the time I put into the project.  http://courses.teresco.org/metal/ (http://courses.teresco.org/metal/)
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on January 31, 2017, 10:14:21 pm
The first lab using our data finished up today, so regular graph generation resumes with tonight's site update (currently in progress).
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on January 17, 2018, 10:47:09 pm
It's time for my Algorithms students to work with highway graph data from TM again.  In order to keep things consistent for the duration of this first lab assignment that involves the graphs, tonight's site update will be the last to generate new graphs until everyone has finished the relevant tasks.  This could be as soon as a week from tonight but as much as two weeks.  As was the case a year ago, this will not stop site updates from happening, meaning the graphs will become inconsistent with what's the in the database as highway data updates are ingested.

(Edit to fix expected timing of resumption of graph generation.)
Title: Re: Temporary Suspension of Graph Generation
Post by: dave1693 on January 18, 2018, 02:09:01 am
I'm excited that this site has academic applications.

As am I. Heck, I'd like to take the class, there's probably new stuff for me to pick up!
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on January 25, 2018, 09:31:41 pm
Everyone is done with the part of the lab assignment that needed graphs from the site, so I'm regenerating them with tonight's site update.  A new assignment is out tomorrow where they'll be using the graphs again, so this should be the only site update with new graphs for about a week.
Title: Re: Temporary Suspension of Graph Generation
Post by: yakra on January 25, 2018, 11:47:18 pm
Does the Temporary Suspension of Graph Generation include tm-master.nmp?
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on January 26, 2018, 06:52:29 am
Does the Temporary Suspension of Graph Generation include tm-master.nmp?

That gets generated, but might not be uploaded when graphs aren't generated.  I will check and can move it over manually if there's interest.
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on January 26, 2018, 10:39:24 pm
Graphs were generated tonight, but I don't plan to generate them again until next weekend.
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on February 03, 2018, 09:54:28 pm
This weekend, my algorithms students are between assignments that use the METAL graphs generated by TM.  New graphs that include recent highway data changes will be generated tonight.
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on February 05, 2018, 10:12:17 pm
Graph generation will continue as part of the usual nightly site update process for at least the next week.  Not sure what next week's lab assignment will bring just yet.
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on February 20, 2018, 11:03:45 pm
First new graphs in about a week will be generated with tonight's update.
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on April 15, 2018, 04:41:32 pm
My students are back at it using the highway data graphs starting tomorrow, so I'm going to suspend graph generation until after their last problem set is due.  The next graph generation will likely not happen before about April 28.
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on April 29, 2018, 08:53:45 pm
My students are back at it using the highway data graphs starting tomorrow, so I'm going to suspend graph generation until after their last problem set is due.  The next graph generation will likely not happen before about April 28.

Still no new graphs tonight, but I'll generate new ones tomorrow (4/30).
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on January 22, 2019, 08:09:30 am
We're into the first Analysis of Algorithms lab this week that will be using METAL graph data, so I won't be generating new graphs until everyone's finished the lab tasks.  That could be as soon as Wednesday evening, but since it's not due for a week, could go through to next Wednesday.
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on January 24, 2019, 01:50:49 pm
Everyone has finished the parts of the lab that use our graph data, so the latest graphs will be generated tonight (which will be the first to include the new marn preview system).

The next assignment, which will definitely be going on for a full week, begins tomorrow, so expect no new graphs to be generated from Friday, 1/25 until at least Friday, February 1.

At some point, I need to look into graph "release" versions that can stay around and be used by METAL users, allowing the most current graphs to be generated on a regular basis: https://github.com/TravelMapping/DataProcessing/issues/177

Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on February 01, 2019, 10:37:05 pm
The assignment using METAL graphs was due today so graphs will be generated tonight and if needed for the next few days.  I'll also suspend generation after Monday night for the next lab assignment, which will be making use of METAL graphs and HDX, but that should last just be a day or two suspension.
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on February 12, 2019, 09:18:23 pm
With the snow-out of one of my lab sections today, the current lab assignment using METAL graphs will be extended into next week.  Expect it to be 7-10 days before graphs are generated again, since I need to make sure nothing changes with the ones they're using.  Sorry if that inconveniences any of our highway data developers.
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on March 07, 2019, 09:32:05 pm
New graphs coming tonight.  The next lab using them in a way that requires consistency for more than a day or two is not for 2-3 weeks.  I'll be generating graphs any time highway data changes warrant until that time.
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on April 08, 2021, 05:21:04 pm
For the first time in a while, I need to suspend graph generation for a bit.  I might be able to include them tonight, but then I'll have a lab set up to run next week for which I need to make sure vertex numbers and the like don't change.
Title: Re: Temporary Suspension of Graph Generation
Post by: Markkos1992 on April 08, 2021, 05:41:06 pm
I am just shocked that you had not posted about this in over two years...   :o
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on April 08, 2021, 05:49:33 pm
I am just shocked that you had not posted about this in over two years...   :o

I haven't used this particular lab (not teaching that course last year or this), but the person teaching it would like to use it, so I'm updating.

Maybe this summer I'll get the graph archives going with maybe monthly snapshots or something that can be used by HDX for situations like this.
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on April 08, 2021, 09:19:57 pm
I will be able to generate fresh graphs tonight.  Now I just have to remember not to do so in the coming days until everyone's finished the lab.
Title: Re: Temporary Suspension of Graph Generation
Post by: michih on April 16, 2021, 09:12:22 am
Has everyone finished the lab now?
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on April 18, 2021, 01:04:26 pm
Graph generation will resume with tonight's site update.  Thanks all for your patience.
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on March 29, 2022, 03:05:17 pm
I will be skipping graph generation for a few days (assuming I remember to do so) for the Siena Algorithms students to have consistent graph data to work with in labs this week.
Title: Re: Temporary Suspension of Graph Generation
Post by: Jim on April 01, 2022, 11:25:29 pm
Updated graphs are being generated tonight and will be updated daily again.