Slow console

[cnotsch]

I don't know if it is my PC or if this is a bug in BC:
If i get let's say 200 lines of text printed into BCs console while playing and i press enter to print the text into the console it takes a lot of time, like it prints all the code you'll never see again because the widow scrolls down. So let's say i've accidently made an error in a torpedo script and some ships fire 100 torps before entering the console, the game nearly crashes (it freezes for minutes) when entering something.

So what I like to know:
Is there a fix,
If not, I had a quite nice idea:

I replace stdout and stderr with something custom that precashes the text into a string or a list or something like that.

My problem: how do I write the key handlers for the console window:
PGUP, PGDOWN to scroll the console
ENTER to execute the code i typed and so on

so i can use the oreginal stdout, stderr to print a single page into the console

[USS Frontier]

Well you could try using the Console Tracker mod.

It will keep "pushing" enter in the console, getting the new text and saving it in a log file.
So with it you won't have these "slow/almost crashing console" problems, and besides that you'll have everything that was printed into the console in a text file.

[cnotsch]

I know i have something like this there i got the first idea from. but i wonder how i can setup the keyhandlers in a seperate module to be able to create scrolling or saving keybindings if you know what i mean

[MLeo]

I have managed to do that partially, but there are a few problems with that.

For example, recursive calls to stdout, causing BC to crash to desktop.
Or, no output to the console, but to the file.

[cnotsch]

could you give me an example of your code?
i thaught i'd do the basics like this:

Code: [Select]

import sys
poldstdoutwrite = sys.stdout.write
sys.stdout.lsTemp = []
def newstdoutwrite(s):
        sys.stdout.lsTemp.append(s)
sys.stdout.write = newstdoutwrite
def writetoconsole(istart, iend):
        if istart >= len(sys.stdout.lsTemp):
                istart = len(sys.stdout.lsTemp) - 1
        if iend >= len(sys.stdout.lsTemp):
                iend = len(sys.stdout.lsTemp) - 1
        for i in range(istart, iend):
                poldstdoutwrite(sys.stdout.lsTemp[i])

[MLeo]

The convention is that you replace sys.stdout.


But I'm working on something else that ought to work even better than I described earlier.

Btw, if you are worried of going out of bounds, do this:

Code: [Select]

l = len(sys.stdout.lsTemp)
istart = istart % l
iend = iend % l

[MLeo]

It would appear that the sys.stdout isn't a plain Python object, but a C++ object (I already knew this, but this is important), as such, you can't overwrite methods on it.


And so far, either it doesn't work, or you don't see anything on the console, and the console doesn't interact, much. At best, so far.



Or you fake pressing Enter on the console on a timer (or thread?).

[cnotsch]

as far as i know there is no problem with the enter button since it calls the sys.stdin.write() method.

[MLeo]

I don't think so:

Code: [Select]

import sys

print dir(sys.stdout)
['close', 'flush', 'getvalue', 'isatty', 'read', 'readline', 'reset', 'seek', 'tell', 'truncate', 'write', 'writelines']

def write(self, s): return s

sys.stdout.write = write
Traceback (innermost last):
  File "<string>", line 1, in ?
AttributeError: write

sys.stdout.lTemp = []
Traceback (innermost last):
  File "<string>", line 1, in ?
AttributeError: lTemp

screendump()


[MLeo]

Also, even something that completely emulates the sys.stdout (and in and err) will trigger this "console doesn't react anymore" behaviour.

Code: [Select]

class ConsolePipe:
def __init__(self, next, name):
#self.__file = nt.open("scripts/Custom/Logs/ConsoleOutput.txt", nt.O_CREAT|nt.O_TRUNC|nt.O_WRONLY)
self.__debu = nt.open("scripts/Custom/Logs/" + name + "Debug.txt", nt.O_CREAT|nt.O_TRUNC|nt.O_WRONLY)
self.__next = next
def __getattr__(self, name):
nt.write(self.__debu, "Accessing: " + name + "\n")
return getattr(self.__next, name)
def close(self):
nt.close(self.__debu)
return self.__next.close()
All this code does, is just "log and pass it on".

[cnotsch]

OK saw it now myself i just assumed that would be also the same as in python 2.5.2 - i only tried it there - , sorry.

[MLeo]

I do admit that this _does_ work in a normal Python Console.

Including 1.5.2.

But this doesn't work in BC, since the console that you see isn't a real Python Console.

I think I need to do some special surgery. ;)

[cnotsch]

i also found out that some areas like for example __builtins__ are write protected, is it possible to open them up?

[MLeo]

You can change __builtins__


But import it first. ;)


I've already overriden __import__ and reload for various stuff (mostly experiments).

But you can't change anything related to the console, and print also doesn't get defined there (it's a real keyword/statement/language construct, like exec).




On the other hand, I have a near working input logger, output "works", except I now need to properly emulate the console input (backspace, delete, up, down, left, right, enter works, kinda, just need to fix the exec statement, eval doesn't work, since print "something" is valid in the console, but not in eval).


[EDIT] The console pipe, with addition of the write and writelines method works, except that the console is then broken since it doesn't process any input anymore.
For example, doing this with it:

Code: [Select]

sys.stdout = ConsolePipe(sys.stdout);print "This gets printed"Will produce:

Code: [Select]

This gets printedBut subsequent input doesn't register.

[MLeo]

I now have (after 3 failed attempts, including one console replacement effort, no shame in failing in this line of business!) a properly working console, output, logger. No (automatic) pushing enter at all. So the only overhead is from writing to disk.

After I finnish the console input logging (involves keeping track of all the inputs and monitoring for Enter keys and remembering which lines are done) I'll mostlikely release it as part of the new UMM, with no ability to turn it off, this is done so that people with script problems can just zip up the latest <<timestamp>>_console.log and put it up with the thread.

Each BC start results in a seperate treble of logs (timestamp ensures both uniqueness and easy searchability).

[Kirk]

Awesome MLeo, this may be one of my most anticipated mods now.

[cnotsch]

nice work I'm looking forward to fix this bug of BC, i really do!
like i may have said bevore this drives me crazy  :lol:
thanks.

[MLeo]

This may not fix (ok, I know it won't fix this) the slowness of the console after lots of printed lines between pressing Enter. But it will dump them all automaticly to a log file.

[cnotsch]

so is it possible to remove the error messages from the console, or even make a button somwhere to enable/disable the error messages?

i do get this right: the error is logged instantly after occuring. ?
if so my idea would solve the problem.

if it is possible to do this with the button you could also make this for standard outputs (e.g. 'print') than will the console run like 'Speedy Gonzales'  :lol: :lol: :lol:

[MLeo]

Not without going into all the scripts that print stuff (not counting bcdebug, since that already dumps everything to a file) and replacing those print statements with other stuff.


The error (or rather, any message) is logged directly to the files, but that doesn't mean it won't dump everything to the console once you press Enter.


But the game is already paused when you open the console, right? So any slowdown shouldn't really matter. And I've already made scripting utilities to dump the console to a file (the Enter pusher works exactly like that, except it pushes automaticly).


Adding GUI to the console isn't a good idea, since that disables either the new gui, or the old gui.
Page up/down won't work, since you can't scroll a "TGPane" (what the Console uses).
Evaluating, but not printing won't work, since it needs the object (the exact object) with the printing ability, at the right location, to even do it's magic (evaluating your input), and will therefor print to the console.
Doing that input yourself, such as in the key handler won't work either, since you can't get at the input box (only visible parts). And certain commands won't work, for example, the most important one, sys.exit(), this will result in a debug window (no error, just the debug window). Which is, sort of, the right behaviour, since sys.exit does raise an exception (it's how it works), except that the real console knows how to deal with it (namely, exiting).

I've tried these things already myself. ;)


TBH, I'm really thinking of leaving out the input entries, a dump command can/will deal with that (if you ask me that is).
What do you all think about this?