2015-06-29

Things I don't like about python: bools indexing into a list

Things I don't like about python

Bools indexing into a list

This is valid code, and does what you expect:

list_ = [ 'A', 'B', 'C' ]
print list_[False]  # 'A'
print list_[True]   # 'B'

... which is neat, but it upsets me a little bit. You index into things with integers, not floats, not strings, and certainly not bools. This smells more like Javascript than Python. If I want to use a bool to index values out I can do that with a dict.

2015-06-23

A Python object to detect when an imported module has had the source code changed

Detecting changes to loaded modules

I'm reworking the web server for my irrigation project. I've already got "kr", which restarts a program whenever it detects the source files have been updated, but first I need the existing web service to close down when there is a change to the source code. For this I implemented the code below, which defines a class. Calling an instance of the class will check the loaded modules (with custom filtering available) for changes to their source code. If a change is found, a callback is made. You can shut down your web server with the callback. The only tricky part is getting the event loop to periodically call the check function. It worked fine for my architecture that doesn't block indefinitely. Later I'll probably add an option for having a background thread do the monitoring.

The file is available on github here.


#!python

#Copyright 2015 Mark Santesson
#
#   Licensed under the Apache License, Version 2.0 (the "License");
#   you may not use this file except in compliance with the License.
#   You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.


# Test with: kr.py *.py -c watch_modules.py


import os
import re
import sys


class ImportedModulesTimestampChecker(object):
    '''
       A class to help detect when one of the source files for a
    running process has changed.
       The returned object should be called periodically to let it check
    the timestamps of all files. There is, as of yet, no multi-threaded
    version.
       Typical use of this module would be to have it track the files
    being actively developed and to call a function resulting in program
    exit (perhaps sys.exit) when a change is detected. This works well
    when combined with "kr" to relaunch the program.

    '''
    def __init__(self, on_difference_fn, filename_filter_fn=None):
        '''
        Takes two parameters:
          on_difference_fn: a function taking the name of the file which
                      changed, which is called when the change is
                      detected.
          filename_filter_fn: a function taking a module's filename and
                      which should return True if the file's timestamp
                      should be observed. If the filename filter is a
                      string, then any filenames containing that string
                      will be observed. If not present, then all modules
                      will be tracked.
        '''
        self._onDifferenceFn = on_difference_fn

        if isinstance(filename_filter_fn, basestring):
            filename_filter_fn = lambda x: filename_filter_fn in x
        self._filenameFilterFn = filename_filter_fn or (lambda x:True)

        self._timestamps = {}

    def __call__(self):
        '''
        Call this periodically to do a check of the timestamps.
        '''
        # Get the timestamps on all matching files in this module's
        # directory. If any have changed, quit.
        all_files = [ x.__file__ for x in sys.modules.values()
                      if isinstance(x,type(sys))
                         and hasattr(x, '__file__')
                         and self._filenameFilterFn(x.__file__)
                    ]
        for module_name in sorted(all_files):
            module_name = re.sub(r'\.pyc$', r'.py', module_name)
            try:
                ts = os.stat(module_name).st_mtime
            except WindowsError:    # TODO: What is the error on Linux?
                logging.exception(module_name)
                pass
            else:
                if ts > self._timestamps.setdefault(module_name, ts):
                    self._onDifferenceFn(module_name)
                    self._timestamps[module_name] = ts


def main():
    import os.path
    import SocketServer
    import logging
    # From an example in the documentation for SocketServer.
    class TinyHandler(SocketServer.StreamRequestHandler):
        def handle(self):
            # Get one line.
            self.data = self.rfile.readline().strip()
            logging.info('Received: %r', self.data)
            out = self.data.lower()
            self.wfile.write(out)
            logging.info('Sending : %r', out)

    class TinySocketServer(SocketServer.TCPServer):
        def __init__(self):
            self._address = ("localhost", 9999)
            SocketServer.TCPServer.__init__( self
                                           , self._address
                                           , TinyHandler )
            self.timeout = 0.5
            self.quit = False
            self._timestampChecker = ImportedModulesTimestampChecker\
                    ( self.on_module_modification
                    , self.module_name_filter
                    )

        @staticmethod
        def module_name_filter(module_name):
            this_mod_name = os.path.basename(__file__).split('.')[0]
            return this_mod_name in module_name

        def on_module_modification(self, module_name):
            logging.info('Module %s was modified, exiting.', module_name)
            self.quit = True

        def run(self):
            logging.info('Listening at: %s', self._address)
            while not self.quit:
                self._timestampChecker()
                self.handle_request()
            logging.info('Quitting.')

    logging.getLogger().setLevel(20)
    tws = TinySocketServer()
    tws.run()

if __name__ == "__main__":
    main()



2015-03-16

Anecdote: That time I had an idea that contributed to the game design

I used to work at Pipeworks Software. We did really cool things like the original XBox demos. I didn't work on Desk Toys, but I did most all of the Ping Pong Ball and Mousetrap room. I did the butterfly flight in Butterfly Garden. I also did part of the boot screen that was used for the original generation XBox. I did the fog, camera paths, and blobby goo.

For Godzilla: Destroy All Monsters Melee" I worked on special effects, rigid body physics, &c. At one point I was coding the results of a monster being thrown into a building. Whenever you destroy a building it is supposed to make the people a bit more angry at you. Unfortunately, when a monster was thrown into a building we had no record of who threw the monster into it. As I contemplated how much work would need to be done to track that information I realized that I could just attribute the destruction to the monster that was thrown. After all, if someone throws me onto an ant pile, the ants are going to get mad at me.

I walked over to a designer and ran the idea past him. It would let the player force some human anger onto their opponent. This wasn't a big deal, but could affect the game as the humans would attack the monster that was doing the most damage to their city. Humans could never kill you, but they could take you down to about 10% health through machine gun fire from helicopters. It seemed like the change would work okay, so I went back and implemented it.

That saved me a few minutes of time and some added complexity to the data structures. And I thought it was a nice feature.

2015-03-11

New Page: My Irrigate Project

I just added a page to the blog introducing my major side project. I've built an Arduino based flow sensor that monitors my irrigation water usage. (It also keeps an eye on my automatic pool filler.) It mails me graphs of water usage every morning so that I can easily see if a sprinkler head has been broken off, thereby saving me money and worry. It also emails me if flow stays on for longer than a set time (in case I've forgotten to turn off water filling the pool). I can check the flow rate in real time from a web page.

I haven't yet packaged up the source, but I'll do that soon. Check it out and leave me comments.