Bridge Commander Central

BC Forums => BC Scripting => Topic started by: cnotsch on June 15, 2008, 10:43:23 AM

Title: Slow console
Post by: cnotsch on June 15, 2008, 10:43:23 AM
I don't know if it is my PC or if this is a bug in BC:
If i get let's say 200 lines of text printed into BCs console while playing and i press enter to print the text into the console it takes a lot of time, like it prints all the code you'll never see again because the widow scrolls down. So let's say i've accidently made an error in a torpedo script and some ships fire 100 torps before entering the console, the game nearly crashes (it freezes for minutes) when entering something.

So what I like to know:
Is there a fix,
If not, I had a quite nice idea:

I replace stdout and stderr with something custom that precashes the text into a string or a list or something like that.

My problem: how do I write the key handlers for the console window:
PGUP, PGDOWN to scroll the console
ENTER to execute the code i typed and so on

so i can use the oreginal stdout, stderr to print a single page into the console
Title: Re: Slow console
Post by: USS Frontier on June 15, 2008, 11:24:05 AM
Well you could try using the Console Tracker mod.

It will keep "pushing" enter in the console, getting the new text and saving it in a log file.
So with it you won't have these "slow/almost crashing console" problems, and besides that you'll have everything that was printed into the console in a text file.
Title: Re: Slow console
Post by: cnotsch on June 15, 2008, 11:56:38 AM
I know i have something like this there i got the first idea from. but i wonder how i can setup the keyhandlers in a seperate module to be able to create scrolling or saving keybindings if you know what i mean
Title: Re: Slow console
Post by: MLeo on June 15, 2008, 01:30:33 PM
I have managed to do that partially, but there are a few problems with that.

For example, recursive calls to stdout, causing BC to crash to desktop.
Or, no output to the console, but to the file.
Title: Re: Slow console
Post by: cnotsch on June 15, 2008, 02:47:33 PM
could you give me an example of your code?
i thaught i'd do the basics like this:
Code: [Select]
import sys
poldstdoutwrite = sys.stdout.write
sys.stdout.lsTemp = []
def newstdoutwrite(s):
        sys.stdout.lsTemp.append(s)
sys.stdout.write = newstdoutwrite
def writetoconsole(istart, iend):
        if istart >= len(sys.stdout.lsTemp):
                istart = len(sys.stdout.lsTemp) - 1
        if iend >= len(sys.stdout.lsTemp):
                iend = len(sys.stdout.lsTemp) - 1
        for i in range(istart, iend):
                poldstdoutwrite(sys.stdout.lsTemp[i])
Title: Re: Slow console
Post by: MLeo on June 15, 2008, 02:59:14 PM
The convention is that you replace sys.stdout.


But I'm working on something else that ought to work even better than I described earlier.

Btw, if you are worried of going out of bounds, do this:
Code: [Select]
l = len(sys.stdout.lsTemp)
istart = istart % l
iend = iend % l
Title: Re: Slow console
Post by: MLeo on June 18, 2008, 06:20:05 PM
It would appear that the sys.stdout isn't a plain Python object, but a C++ object (I already knew this, but this is important), as such, you can't overwrite methods on it.


And so far, either it doesn't work, or you don't see anything on the console, and the console doesn't interact, much. At best, so far.



Or you fake pressing Enter on the console on a timer (or thread?).
Title: Re: Slow console
Post by: cnotsch on June 19, 2008, 09:36:20 AM
as far as i know there is no problem with the enter button since it calls the sys.stdin.write() method.
Title: Re: Slow console
Post by: MLeo on June 19, 2008, 11:55:23 AM
I don't think so:
Code: [Select]
import sys

print dir(sys.stdout)
['close', 'flush', 'getvalue', 'isatty', 'read', 'readline', 'reset', 'seek', 'tell', 'truncate', 'write', 'writelines']

def write(self, s): return s

sys.stdout.write = write
Traceback (innermost last):
  File "<string>", line 1, in ?
AttributeError: write

sys.stdout.lTemp = []
Traceback (innermost last):
  File "<string>", line 1, in ?
AttributeError: lTemp

screendump()
Title: Re: Slow console
Post by: MLeo on June 19, 2008, 12:23:00 PM
Also, even something that completely emulates the sys.stdout (and in and err) will trigger this "console doesn't react anymore" behaviour.

Code: [Select]
class ConsolePipe:
def __init__(self, next, name):
#self.__file = nt.open("scripts/Custom/Logs/ConsoleOutput.txt", nt.O_CREAT|nt.O_TRUNC|nt.O_WRONLY)
self.__debu = nt.open("scripts/Custom/Logs/" + name + "Debug.txt", nt.O_CREAT|nt.O_TRUNC|nt.O_WRONLY)
self.__next = next
def __getattr__(self, name):
nt.write(self.__debu, "Accessing: " + name + "\n")
return getattr(self.__next, name)
def close(self):
nt.close(self.__debu)
return self.__next.close()

All this code does, is just "log and pass it on".
Title: Re: Slow console
Post by: cnotsch on June 19, 2008, 12:52:34 PM
OK saw it now myself i just assumed that would be also the same as in python 2.5.2 - i only tried it there - , sorry.
Title: Re: Slow console
Post by: MLeo on June 19, 2008, 01:06:42 PM
I do admit that this _does_ work in a normal Python Console.

Including 1.5.2.

But this doesn't work in BC, since the console that you see isn't a real Python Console.

I think I need to do some special surgery. ;)
Title: Re: Slow console
Post by: cnotsch on June 20, 2008, 02:20:40 PM
i also found out that some areas like for example __builtins__ are write protected, is it possible to open them up?
Title: Re: Slow console
Post by: MLeo on June 20, 2008, 02:40:02 PM
You can change __builtins__


But import it first. ;)


I've already overriden __import__ and reload for various stuff (mostly experiments).

But you can't change anything related to the console, and print also doesn't get defined there (it's a real keyword/statement/language construct, like exec).




On the other hand, I have a near working input logger, output "works", except I now need to properly emulate the console input (backspace, delete, up, down, left, right, enter works, kinda, just need to fix the exec statement, eval doesn't work, since print "something" is valid in the console, but not in eval).


[EDIT] The console pipe, with addition of the write and writelines method works, except that the console is then broken since it doesn't process any input anymore.
For example, doing this with it:
Code: [Select]
sys.stdout = ConsolePipe(sys.stdout);print "This gets printed"Will produce:
Code: [Select]
This gets printedBut subsequent input doesn't register.
Title: Re: Slow console
Post by: MLeo on June 23, 2008, 10:16:32 AM
I now have (after 3 failed attempts, including one console replacement effort, no shame in failing in this line of business!) a properly working console, output, logger. No (automatic) pushing enter at all. So the only overhead is from writing to disk.

After I finnish the console input logging (involves keeping track of all the inputs and monitoring for Enter keys and remembering which lines are done) I'll mostlikely release it as part of the new UMM, with no ability to turn it off, this is done so that people with script problems can just zip up the latest <<timestamp>>_console.log and put it up with the thread.

Each BC start results in a seperate treble of logs (timestamp ensures both uniqueness and easy searchability).
Title: Re: Slow console
Post by: Kirk on June 23, 2008, 11:53:48 AM
Awesome MLeo, this may be one of my most anticipated mods now.
Title: Re: Slow console
Post by: cnotsch on June 23, 2008, 04:51:57 PM
nice work I'm looking forward to fix this bug of BC, i really do!
like i may have said bevore this drives me crazy  :lol:
thanks.
Title: Re: Slow console
Post by: MLeo on June 23, 2008, 04:58:17 PM
This may not fix (ok, I know it won't fix this) the slowness of the console after lots of printed lines between pressing Enter. But it will dump them all automaticly to a log file.
Title: Re: Slow console
Post by: cnotsch on June 23, 2008, 05:08:50 PM
so is it possible to remove the error messages from the console, or even make a button somwhere to enable/disable the error messages?

i do get this right: the error is logged instantly after occuring. ?
if so my idea would solve the problem.

if it is possible to do this with the button you could also make this for standard outputs (e.g. 'print') than will the console run like 'Speedy Gonzales'  :lol: :lol: :lol:
Title: Re: Slow console
Post by: MLeo on June 23, 2008, 05:50:34 PM
Not without going into all the scripts that print stuff (not counting bcdebug, since that already dumps everything to a file) and replacing those print statements with other stuff.


The error (or rather, any message) is logged directly to the files, but that doesn't mean it won't dump everything to the console once you press Enter.


But the game is already paused when you open the console, right? So any slowdown shouldn't really matter. And I've already made scripting utilities to dump the console to a file (the Enter pusher works exactly like that, except it pushes automaticly).


Adding GUI to the console isn't a good idea, since that disables either the new gui, or the old gui.
Page up/down won't work, since you can't scroll a "TGPane" (what the Console uses).
Evaluating, but not printing won't work, since it needs the object (the exact object) with the printing ability, at the right location, to even do it's magic (evaluating your input), and will therefor print to the console.
Doing that input yourself, such as in the key handler won't work either, since you can't get at the input box (only visible parts). And certain commands won't work, for example, the most important one, sys.exit(), this will result in a debug window (no error, just the debug window). Which is, sort of, the right behaviour, since sys.exit does raise an exception (it's how it works), except that the real console knows how to deal with it (namely, exiting).

I've tried these things already myself. ;)


TBH, I'm really thinking of leaving out the input entries, a dump command can/will deal with that (if you ask me that is).
What do you all think about this?
Title: Re: Slow console
Post by: cnotsch on June 24, 2008, 02:49:25 PM
nice idea there.

Quote
ut the game is already paused when you open the console, right? So any slowdown shouldn't really matter.

that is true but what if you have to wait 15 minutes? (wich can in extreme situations be caused by playing 3 min.

Quote
Adding GUI to the console isn't a good idea

i didn't mean exactly into the console i know this would be extremly difficult, but somewhere into the main menu.

but after all i really look forward to get my hands on it  :)
Title: Re: Slow console
Post by: MLeo on June 24, 2008, 04:03:09 PM
15 minutes? That's way too much, and I never even managed that unless I accidently logged a (near) endless loop. ;)

What are you doing anyway?

To actually do this might be rather tricky.
I'll see what I can do, but you will have to navigate UMM to reach this setting, I was hoping to evade creating a gui, since it means I will then also have to provide an "off" setting for it. :(
It isn't the GUI work, that's not hard, it's all the other things I would then have to consider.
The off setting means that people mostlikely will turn it off, which in effect means that when they have an error, they won't even bother looking for a log. :(
Title: Re: Slow console
Post by: cnotsch on June 24, 2008, 05:57:35 PM
then release a stock version for 'those people who turn it off and forget it'
and a new version for the porfis... maybe

not that important... i think about scripting a new console GUI maybe you could tell me where to plug in a command for passing the output... i do have a good idea but i may need some time to complete my actual work first.
Title: Re: Slow console
Post by: MLeo on June 24, 2008, 06:07:41 PM
Like I said in a previous post, a new Console GUI won't work, I've tried that already. ;)

You need the exec keyword/function for this.
But doing so in an event handler other than the console will possibly result in an exception.
For example, sys.exit most certainly won't work.

And you can't enter new commands into the console, unless you do it the slow and totally unadvised way via keypress emulation on the console (which is what the console tracker, not mine, does, it emulates a keypress).

As for a hook, you can get a MainWindow for the console through:
App.TopWindow_GetTopWindow().FindMainWindow(App.MWT_CONSOLE).
You can replace (or rather, hide) the original GUI and then you don't need to worry about the ` key, since it just appears then.
Title: Re: Slow console
Post by: cnotsch on June 24, 2008, 06:16:57 PM
my idea was to create a complete new console from scratch

but if calling the exec statement by a button would result in a error (i really hope not but i won't wonder if not because that would be one of the useful python basics and we all know most of them dont work in BC  :lol: :( :? :x)

otherwise i see no problem.

EDIT: Oh i forgot that with the input i'd realize with a paragraph (is it called like that?) i already used this in my new ATM quite sccesfully
Title: Re: Slow console
Post by: MLeo on June 24, 2008, 06:31:53 PM
Yeah, but if you evaluate exec("import sys;sys.exit()") (or even just sys.exit()) then it will result in an error which you (nor me) can not handle properly, and only the Console itself can do that. It does work for most stuff, but not for one of the most usefull utilities in BC (sys.exit()).


exec is actually a keyword in Python, meaning, so much baked in, it would be rather painfull to remove it, unlike things like regex and xml, which are "just" standard extension modules in Python, which were removed. And it's usefull.


The reason why the Console can handle it, is because it captures all the exceptions Python generates (otherwise it would leave the console in an unusable state). One of them is the one sys.exit generates (SystemExit to be exact), now, if you want to really exit Python through that (and you must), and you do your own console, you would just not capture the exception (or rethrow it), but because you are working in an event handler, in BC, you will either ignore it (don't exit BC), or you will allow the Debug Window to pop up with no errors shown (other than the one you print).
But because we are in the console, a C++ side object, it can notice the exception and handle it by calling it's own, internal, exit procedure, which results in the neat exit of BC.
Title: Re: Slow console
Post by: cnotsch on June 24, 2008, 06:41:50 PM
if it is just that one statement i think that would NO problem!
firstly i have NEVER used it so far, secondly you could filter it out of the text and call it manually in a script.
Besides this should be no replacement it should rather be a second console, main idea was to include a open/save file/module possibility meaning to create a console as near as possible to a stock python interpreter
Title: Re: Slow console
Post by: MLeo on June 24, 2008, 06:53:38 PM
if it is just that one statement i think that would NO problem!
firstly i have NEVER used it so far, secondly you could filter it out of the text and call it manually in a script.
I use it all the time, I recommend it to people all the time in case of BSOD, especially the last one (also for myself, I do quite a lot of things these days that happen on startup of BC). Pretty needed wouldn't you say? Also, console utilities such as dumpConsole, screenshot and, wait for it, exit(), the last one, of course, uses sys.exit().

Calling it manually from a script has the same effect, it's the same thing that's calling it, an event handler.


Quote
Besides this should be no replacement it should rather be a second console, main idea was to include a open/save file/module possibility meaning to create a console as near as possible to a stock python interpreter
Which exec is. In fact, if you pass in the __dict__ of the __main__ module, it is the "stock python interpretter".

With open/save file/module possiblity, you mean save and load games? That's already possible, it's called Quick Save/Load Game.
Title: Re: Slow console
Post by: cnotsch on June 24, 2008, 07:03:36 PM
Quote
Pretty needed wouldn't you say?

nope not at all i don't know why it is people saying BC can't be exited with the windows command ALT+F4??? it has worked on every BSOD - and i had hundreds - eyery crash and simply every error (fullscreen & windowed)...

Quote
With open/save file/module possiblity, you mean save and load games?

not at all i mean you can load/read/write/save multiline scripts/modules ingame like in editor!
EDIT: of course the save/load would only work in scripts/Custom/...

besides would not the console handle the exception (i thaught those were global) if it still exists? i have no intention in touching the console i just want to get the text you write ito the log to be printed into my new window
Title: Re: Slow console
Post by: MLeo on June 24, 2008, 07:21:00 PM
As soon as you do exec in anything other than the console, and it produces a SystemExit exception, you are basicly dead in the water with either no exit of BC, or a debug window.
The reason why the console doesn't do that, is because you aren't in the console.

You basicly have 2 things, either the console, dynamic input, or modules, where there could also be dynamic input, but those are still modules, in fact, the console is also a module (__main__), but it still is a special case since most of that is done by the exe, including handling the SystemExit exception.



Alt+F4 never worked for me in BC. It does work on the debug console though.



I also don't think that a paragraph is the most usefull editor you can have for files, no scrolling (not easily anyway, well, it's possible I suppose, but I'm unsure of the effect) for one.
I'd rather edit them in notepad. :P
Title: Re: Slow console
Post by: cnotsch on June 25, 2008, 01:17:35 AM
Quote
I'd rather edit them in notepad.

i disagree on that i think it is very helpful to be able to check your script instantly without restarting BC all the time, this totally sucks :lol:, i now get around that with making sure my module completely resets on calling its exit() so i can reload(module) and init() it again. but that is also a bit of... work all the time and I'm a LAZY person  :lol:

maybe I find a solution there but for that I need free time where I do not have a big bunch of work planned for, like my ATM is now.
Title: Re: Slow console
Post by: MLeo on June 25, 2008, 08:29:05 AM
I do that all the time (when possible of course, some scripts of mine don't like being unloaded).

Just run in Windowed mode and alt+tab.

You will have to type reload anyway.
Title: Re: Slow console
Post by: cnotsch on June 25, 2008, 02:34:40 PM
why in windowed i don't get it the only difference i see is that silly console breaking the game on every little error, alt+tab, alt+f4, windows, (+d),... all that works in fullscreen too.... am i doing something totally right when intalling windows or why can it be that those features seem only to work on (all) my pcs....

anyway, yeah it is a nice to be able to reload but it would be helpful to be able to write a quick loop with an if and something else ingame without exiting it (or writing it into my libary and reload it)
Title: Re: Slow console
Post by: MLeo on June 25, 2008, 02:40:46 PM
Maybe I ought to make a screencapture of my work flow (or something like that).

Because I can do this:


Start BC
Alt tab
Edit a file
Back into BC
reload(......)
Watch what happens
Alt tab
edit a file
etc,
etc,
etc.
Title: Re: Slow console
Post by: cnotsch on June 25, 2008, 02:45:20 PM
like i said would be MUCH faste to simply edit and press a button ^^ like i said i am veeeeery lazy  :lol:

besides is there only one single time where you don't respond in at least 2 min.? how do you do that? sorry for getting of topic but i had to ask
Title: Re: Slow console
Post by: MLeo on June 25, 2008, 02:50:15 PM
like i said would be MUCH faste to simply edit and press a button ^^ like i said i am veeeeery lazy  :lol:
Yes, that might be faster, but not by so much, since a TGParagraph is less than what you get from notepad (in terms of ease of use). And things like syntax highlighting makes scripting (or any kind of programming) faster. To make matters worse, I'm even thinking of creating an Eclipse plugin for BC specificly, with (if possible) intellisense. ;)

Quote
besides is there only one single time where you don't respond in at least 2 min.? how do you do that? sorry for getting of topic but i had to ask
I do at times, like when I'm asleep, or at dinner, or at work (or school) or doing something else. But I can multitask as it were when I'm behind my computer.



Btw, the debug console is far more usefull than a plain BSOD, it's even possible to fix stuff while in the debug console, say, you forgot to include an import in a module (and a function didn't work because of it), you can fix that in the console, and then type resume!
Title: Re: Slow console
Post by: cnotsch on June 25, 2008, 03:00:18 PM
Quote
I'm even thinking of creating an Eclipse plugin for BC specificly, with (if possible) intellisense.

what is that?

Quote
Btw, the debug console is far more usefull than a plain BSOD

i never got a bsod without the possibility to check the console for errors

Quote
it's even possible to fix stuff while in the debug console

didn't know that how do you do that?
Title: Re: Slow console
Post by: MLeo on June 25, 2008, 06:15:34 PM
Eclipse is just one of the biggest, expandable, Integrated Development Environment (IDE) on the world. Maybe except Emacs, but it looks like Eclipse is going to be the next Emacs.


The reason it's more usefull, is because it practicly gives us a step by step debugger.

We can fix stuff by replacing the faulty method from another file (that is stateless, ie. no data, such as App.py, Foundation.py, for example, is statefull, since it stores data).

Then, as soon as you have replaced the method (an import/reload and assignment away) then you can type in resume and it will continue till the next error.
Or you replace a variable (or set a variable) where you forgot to initially.

I even deem it possible that one day we find the magic trigger so that we always have the debug console at our fingertips, meaning, true, live, changing, instead of the current paused games. ;)
Title: Re: Slow console
Post by: cnotsch on June 26, 2008, 04:10:07 PM
Quote
Eclipse is just one of the biggest, expandable, Integrated Development Environment (IDE) on the world.

oooops i never heard of that one....  :oops:
i don't exactly understand your last post but i think i will understand when you release it someday ^^