Data analysis software for mapping based on log files

[dnb]

I have a few ideas I'd like to kick around here, and given we're all trying to do basically the same thing then sooner or later we'll all have the same ideas so if these ideas are already coded, then please point me in the right direction!

Alternatively, you can tell me I'm daft and it'll never work.  

I have thought of ways for most of this to be automated, requiring little or no user intervention.  (Details are deliberately a little sketchy here, otherwise I may lead you down certain ways of thinking and I prove myself right by circular argument... )

I'd like to take a log file and bin it according to the maps in the appropriate MSQ file.

This should allow estimates to be made on VE accuracy given certain constraints, application of certain filters and a statistically valid number of samples per bin.  We can then derive and apply to the MSQ a set of map corrections that should be of a much higher quality than autotune could ever manage.

It would also allow an estimate of drift WRT intake temperature (This was a serious problem for me on the TVR last time) and allow the value of the correction parameter to be calculated.

We could look at knock sensor noise data in a similar way.  The first thing is to estimate the divide between MBT based and knock based operation, then apply appropriate corrections to the knock based areas.  Again, there are filters that must be applied to the data.

I can't quite see my way round the next ideas, but I'm sure there is a way

If we use the 2ndary trigger to log vehicle road speed (is there a way to do this?) then we can construct a much better lambda map based the position of the main cruise speeds (say 30, 40, 60 and 70) in the map.  We can also look where in the map deltaTPS is appearing.  This can then be correlated to differences between lambda target and actual lambda, given that we know what the EGO correction is up to on average in these cells.  This leads to real-world estimates of corrections for acceleration enrichment.

The addition of vehicle speed in the logs adds even more potential - we can then do meaningful fuel use analysis etc.

Most of this can be done in excel, but it takes many hours work in setting it all up.  It would be much nicer if this could take minutes.

[LT401Vette]

I hear you
MegaLogViewer does a great job with the VE tables, in the MegaSquirt community MSTweak3000 has long since been put to rest. But there is a whole lot more opportunity. Ultimately it is possible to build an application that can tune most aspects. Ideally with full connectivity even things like true measurement of injector open time is not much work to implement.
This got me going a while back and I set out to write it in Oct 2006, this has eaten into the time that may have otherwise gotten MegaLogViewer even further by now, but this new core platform is going to allow all these things to be realized. I am near an Alpha release, by years end. The initial release will be little more than the next generation MegaTune, MegaTune functionality and compatibility with a richer UI, custom dashboards, wizards, and a plug-in interface for extensions. But once that is stable, the core allows for so much flexibility to build all the automated tuning pieces.

Phil
http://www.efianalytics.com/MegaLogViewer

[rob@vems.co.uk]

Awesome stuff Phil,  I have to say thanks to you for MegaLog viewer, its helped me out more times than I can count. And I hear nothing but praise for it from the people I advise to use it.

I see that your page has a Buy MegaLogViewer Now! link, is this for the new software? https://www.efianalytics.com/register/register.jsp?appName=MegaLogViewer

What will the plug-ins need to be written in?

[GintsK]

I much say big thanks for Phil too! I use this piece of software every day!
with a some hacks in vems3.ini i got VEanalyser working for VEMS too. Not so good as for MS, but for usable 16x14 Alpha-N table 1 hour was enough.
What we must do for full adapting MLV for VEMS?

[dnb]

Hi Phil!

I've made quite a bit of progress since I posted the thread.  All of the fuel side is now automated to a reasonable extent, including estimating injector parameters (although I'm not too happy with the answers yet - I need more resolution in the logs for certain areas...)

Next stop is the obvious one - an interface with a dyno to do ignition timing optimisation

[LT401Vette]

Compatibility should be easy. I just need to see some logs & msq's.

The latest version of MLV has all fields references pulled out into an external properties file, so you are no longer required to use the MegaSquirt field naming convention.

The msq is the other piece, there are still a few references in there to specific bins from MS1 and MS1.

The "Buy MLV" is for MLV, it is shareware now. There is no disabled functionality though, just a popup nag to register. To buy will get a registration to get rid of the nag. The nag is only meant to be a reminder, not much of an annoyance I hope.

[rob@vems.co.uk]

Compatibility should be easy. I just need to see some logs & msq's.

The latest version of MLV has all fields references pulled out into an external properties file, so you are no longer required to use the MegaSquirt field naming convention.

The msq is the other piece, there are still a few references in there to specific bins from MS1 and MS1.

The "Buy MLV" is for MLV, it is shareware now. There is no disabled functionality though, just a popup nag to register. To buy will get a registration to get rid of the nag. The nag is only meant to be a reminder, not much of an annoyance I hope.

Here's a log and MSQ for your perusal, let me know if you need more:
http://koabi.lazyslacker.com/VEMS/logs/20071201megasquirt.msq
http://koabi.lazyslacker.com/VEMS/logs/20071201datalog.xls

Shareware nags are fine by me, I was wodnering if you were planning on making the license mandatory at any point?

[LT401Vette]

Thanks, those will help. Do you know if there is much variance with different firmwares? Actually is it common to have various firmwares for VEMS, or is a single strain that just moves forward?

I don't have any plans to make the registration mandatory. I would guess that would narrow adoption anyway, many people would probably stay on the earlier and it would sort of be hanging out there with no support. The people paying are typically just doing it to show their appreciating and return the favor more than to get rid of the nag. That is why I mean for the nag to be more of a reminder than a pita.

However with that said, I do see some new larger features in the future that may not be enabled without a registration.

[rob@vems.co.uk]

The firmwares tend to build on each other - the output from the datalog stays pretty consitent.  The .ini file for MegaTune changes quite a bit inbetween firmware releases - mainly so that we can put more fields in, and remove the redundant ones.  I imagine that

I see no reason why you couldnt charge for the "pro" features.  The bigest complaint I get from tuners is the poor keyboard support - the ability to select a group and either enter a number directly from the keyboard or use the arrow keys.  I did some work on a table editor:

http://www.vems.co.uk/VEMSTech/table.zip

Cheers Rob

[LT401Vette]

That works pretty good.

Thanks, that is good feed back. I was actually putting better keyboard in, but more for menu functions. The table editors would really benefit too. Expect that to be in an upcoming version.

[rob@vems.co.uk]

Here's the keys:
    *  Ctrl-L allows editing of the load table.
    * Ctrl-R allows RPM table editing.
    * Ctrl-C copies the content of the selected cell.
    * Ctrl-V pastes the copy into the selected cell or group of cells.
    * F9 changes to VE table.
    * 10 changes to Ignition table.
    * 11 changes to Lambda table.
    * Select a group of cells in the main table:
    * Page Up to increase the cells' value.
    * Page Dn to decrease the cells' value.
    * Type a numeric value to set the selected cell's
    * Bounds checking for keyed entry (0-255 for VE, xx.(25|50|75) for ignition, 0.7->1.x for lambda) Lambda does not scale correctly - please advise
    * Small increment with q|+|Pg Up
    * Small decrement with w|-|Pg Dn
    * Large increment with e|Ctrl +|Ctrl Pg Up
    * Large decrement with r|Ctrl -|Ctrl Pg Dn
    * Paint value - Ctrl Up, Down, Left, Right

[cliffb75]

Something I found useful in another aftermarket system was the ability to lock cells that you have mapped and then interpolate between them. It makes getting the intial shape of a maps quite quick and easy, and avoids accidentally adjusting a 'real' value

Also, for group selection, typing a fixed value is useful, and so is adding/subtracting an offset(ofter useful for ignition or boost related tables) and multiplying/dividing (for fuel tables).

[gunni]

To update on this.

assuming your running normal petrol I found that using in Megalogviewer to use VE_analyzer with Vems.
Wideband O2-AFR should be "Dynojet linear"

that will result in Lambda 1 = AFR(WBO2) to be 14.7
and will thus allow the VE_analyzer to be used,

When I tested I had it disregard MAP below 35kpa which is sort of overrun and Vems can show some very high numbers in the lambda field  that will mess with the results. Running it over a very recent log it showed exactly the results I expected.

Tomorrow I´m changing
Inj open time @ 13.2v (us) to 0 from the 1008 it is now, which is wrong since I have a diode in place.
So using the VE Analyzer I´m expecting fuel to be sorted in less then an hour.

[LT401Vette]

Gunni,
Thanks for bringing this thread back to life. Somehow it completely fell off my radar!
I think I have just been too heads down with TunerStudio.

So is the WB sensor consistent with VEMS? It is always the same and matches the Dynojet?

[gunni]

Log files have Lambda values directly in them and the wideband is integrated in the VEMS no outside voltage stuff.

It would be best if the VE analyzer could just use the "lambda" logged value as well as reading the msq lambda table.
As well, some vems log it as "lambda" while other version "Lambda" .