Python Q: Variables Ranging Over Variables

Factual Questions
1

In Python, can I have a variable ranging over names of variables?

For example, could I write something which checked to see whether the value of any variable with a name beginning with the string “substance” is, say, 0?

I asked about Python because that’s what I’m working on learning right now, but I’m curious to know if you can do this in any language (or can you do it in all of them?) and if so, how does it work in that language? (Is it a straightforward part of the language, or do you have to play some tricks to do something like it?)

-FrL-

A related (somehow) and probably simpler question: Is there a way to check whether a particular object exists or not? For example, if I delete an object, can I later have a line or block which checks to see whether that object still exists, where that line or block will return a “no” value because that name for an object no longer names any object?

I can do this with a flag variable set to 1 upon deletion of the object, but due to the fact that Python’s variables are not global, this is difficult if not possible to make work in many cases. I can set a variable to be global, but I hear that’s not recommended. I can use a list instead of a single valued variable as lists are global, but that seems arcane.

2

…

3

If a language had a feature like you’re describing, it would violate the principle that changing a variable name has no effect on program execution as long as there aren’t two variables of different types with the same name in the same scope (and most interpreters wouldn’t even run a program like that). If this seems like a good idea, you’re probably writing bad code.

As far as checking for the existence of objects, probably not. Any interpreter’s internal naming scheme for objects is going to use its own identifier system (object IDs in Java, addresses in C++) and I don’t know of any that will guarantee that internal names never get reused. So you could see whether the object ID that your object had is still assigned to something, but you can’t be guaranteed that it’s the same object.

4

Yeah, it would be pretty ugly. You want to think of variable names as mere binders, insignificant precisely the way the bound variable in, say, “the integral of x^2 dx from 0 to 3” is insignificant.

That having been said, Python does have significant capabilities in this vein, ugly or not. The word to search for is “introspection”, and more specifically, I think the “getattr” function is relevant to what you’re asking about.

5

What you actually want to do is use a hash table for this purpose (Python calls them mappings). That’s what they’re for.

6

Sure, if all you’re looking for is the functionality of a user-definable map from strings to values, then a hashtable is all you want. But if you specifically want to access the particular map which automatically goes from the names of the actual “Python-level” variables to their values, then you’ll need the introspection features.

7

Agreed. But there’s almost never any reason to actually do that.

A huge number of programming questions are in the form “How do I accomplish this ridiculous thing?” The questioner never states what actual problem he’s trying to solve, because he’s already thought of a solution. If he did state the actual problem, then the much more sensible solution would be offered instead. That’s what I was trying to do, in my admittedly somewhat snotty way. :slight_smile:

8

Yes, that’s an excellent point.

9

Sure.

If you have an object with attributes named ‘substanceX’, ‘substanceY’, and ‘substanceZ’ and you have a user of this object that wants to determine if any of these variables are not zero, then you can use the dir() function. Use dir() to iterate through the attributes and use string compares to find any attributes named ‘substance.*’. This code will work even if the object adds attributes ‘substanceA’, ‘substanceB’, and ‘substanceC’.

You can also implement the opposite case. You have an object with an attribute ‘substance’ and you want users of the object to be able to refer to this same variable using a variety of names (e.g. ‘substanceX’, ‘substanceY’, ‘substanceZ’). For this case, your class should implement getattribute() and setattr() to alias all of these attribute names to the real attribute. Using getattribute() can be a bit tricky because it is called for every attribute, so look for a few examples that are similar to your problem. I found examples in the ‘Python Cookbook’ and on the ActiveState programming network. Also, your class will need to be derived from object (in other words a new style class).

I used this solution for a SIP stack that I implemented. SIP is a protocol similar to HTTP where packets are made up of various headers, but the legal headers are not known ahead of time. So that stack allows applications to add any header to a packet, but using the syntax as if they were full-fledged attributes.

If you are going to use this technique, also consider using getitem(), setitem(), and delitem() instead since it has the same power with a less surprising syntax.

As others have said, google for ‘python introspection’. The first two or three hits are really useful.

I’m not positive about this answer, but I think you could use a weak reference (weakref). This lets you keep a reference to an object without preventing it from being deleted and garbage collected. You can then use the weakref to check the state of the object. Again, I haven’t used weakref in Python, but my reading of the docs implys this would work. Google ‘python weakref’ and take the first hit.

Finally, I too am interested in hearing the exact problems that you are solving. These are esoteric constructs and possibly there are easier ways of solving the problems.

10

I am not trying to solve any problem that requires anything more than basic beginner’s stuff, I’m sure. It’s just that my ignorance of basic beginner’s stuff has given my mind the freedom to roam out to crazy land.

My approach to learning has been (or so far has tried to be) “Think of a problem, imagine how I would solve it step by step if I weren’t trying to write code but rather just English instructions, then go find (in the documentation or in online tutorials) whatever command or set of commands in this language can help me do things the way I’ve just imagined.”

But this has led me to propose (what are apparently ugly, bad and ridiculous) problem solving methods like having variables refer to names of other variables and so on. :slight_smile: *

Anyway, the thing I was trying to do is this. I wanted to have a program in which there are volcanos, and batches of substances like “baking soda,” “vinegar,” and “water,” such that if the user puts both baking soda and vinegar in the volcano, then it explodes. No problem there. But I wanted to make it such that the user can make new volcanos, each with its own name, and (w)hip up new batches of substances, each with its own label (created by the user), such that the user could have the program, for example, put the third batch of baking soda into the third volcano, the first batch of vinegar into the second volcano, and so on, all using user created labels. Probably I want to have those labels listed as menu items, rather than requiring the user to type them in anew at each use.

(The purpose of all this? I’m just trying to teach myself OOP. It’s an alien world to me. Remember, before this, my experience was with BASIC in the mid 90’s as a high schooler.) (Also, to be clear, I have no practical purpose for learning this. It’s just for the purpose of, so to speak, “improving the mind.” In other words, I’m just noodling around.) (BTW for all I know this is a terrible kind of task to try to work one’s OOP mojo on. If that’s the case, please let me know, and describe for me what kind of task would better for OOP pedagogy.)

An easy way to do what I described, if you’re not limiting yourself to what’s easy to code would be, of course, to have each object’s name (the variable bound to each object) be whatever the user labeled that object, then when the user uses these labels, the program can take those label-uses and translate them directly and easily into the variable names bound to the relevant objects.

But that is, I gather, not the right way to think of the task for coding purposes.

I am starting to glean (from this thread, and from just thinking about it) that I should be able to use a dictionary centrally for this purpose somehow.

I know this probably seems completely idiotically simple to you guys. And the thing is, I’d rather not have someone just tell me exactly how to do what I’m describing. (It is to avoid this possibility that I refrained from describing the actual task.) But I guess it would be helpful, if someone were kind enough to help, for that person or persons to give me some kind of “pointers,” for example to name and if necessary briefly describe some concept or set of instructions or use of instructions that I might be missing, and that might give me the key to solving the problem I’ve created for myself.

Or if you just want to tell me how to do it, put it in spoiler boxes. :slight_smile:

Or if you think this is something I damn well ought to be able to figure out for myself even as a total beginner, then leave it at that, and after another couple of hours of thinking/coding, I’ll surely have it figured out.

Thanks for the comments on this thread. They’ve been illuminating.

-FrL-

11

By the way, regarding the second question in my OP (about checking for the existence of an object) for the record I wish to note that I know (or anyway, to fully disclose, I just recently realized) it would be quite easy to check for the existence of a particular user created label–for example, just append each newly created label to the end of a list of labels and use the “if x in listname” construction. And(I just recently realized that) this would probably do whatever work I wanted to do when I was wondering how to check to see whether there exists an object with a particular name.

12

I am by no means the SDMB’s foremost expert on OOP, and I’ve never used Python, but it seems to me that [spoiler]You have two types of objects: volcanos, and the stuff that goes in them. What they’re called internally doesn’t matter to your user, but there is a name that the user will want to use to refer to it. Again, I don’t know Python syntax, but you might have a volcano with the following attributes:

myVolcano.name=“Frylock’s Volcano”
myVolcano.bakingSoda=3 (i.e. it currently contains 3 spoonfuls of baking soda)
myVolcano.vinegar=0
myVolcano.eruptions=4 (i.e. it has erupted 4 times already)

Anyway, my point is, the name of the volcano could be an attribute just like anything else, and completely separate from the name of the variable. Then, to check for a particular name, make sure all your volcanos (and perhaps all your boxes of baking soda and vinegar as well) are in one array, so that you can iterate through them, looking for a volcano with a particular name.
[/spoiler]

If everything I’ve said is totally wrong, someone please correct me, and Frylock and I will both learn something. Also, I recently found out about Google Code Search, so you might poke around there and see if it helps.

http://www.google.com/codesearch

13

I think he also wants the user to be able to create user-definable substances, in addition to “baking soda”, “vinegar”, whatever.

14

I was thinking about it.

Ultimately I want the “explosion” (which, btw, is supposed to destroy the volcano) to be triggered by properties of baking soda and vinegar, not just by their bare presence together. I wanted other property pairs to have other effects as well. And I was thinkjing about having a way for the user to create new substances with new combinations of properties,

This is all on the way to a slightly more worthwhile project–an “Alchemy” game, one formally equivalent to a kind of maze but where the idea is to glean the properties of various “substances” and figure out how to mix them and cook them etc in order to produce a particular substance. The most difficult “maze” would be the one that turns lead into gold, of course. (Or maybe one that makes the Philosopher’s stone, come to think of it.)

But I as trying to get this much simpler exercise designed first as a way to test various concepts that would be used in the more complicated one.

Right now the only user-definable attribute of the substances I want to start with is the label the user assigns to it. (“Baking Soda Batch number 1”, or “My first powder,” or “Bob,” or whatever.) (I’ll deal with explicit properties of substances later.) Assigning the name attribute is easy. It was using that name attribute to do things to the object that I was stumbling over. Like I said, I think (I haven’t had a chance to actually use the Interpreter since last night’s post) I have an idea now that I need to use a dictionary for this. Does that sound right?

-FrL-

-FrL-

15

Just for the record, what you proposed was none of those negative things. Those are legit solutions to problems. It just seemed like they would be more complicated solutions than what you needed.

I’m glad you explained your problem more, since I misunderstood the scope of what you wanted to do. The techniques of getattr(), dir(), getattribute(), etc. work best when you are writing code to be used by other developers (e.g. a library). Since your user’s are actual users of the program (and thus they won’t see the code), you don’t need the techniques.

Anyway, your idea for a program/game sounds pretty cool. I’d like to learn more about it when you finish. I think it is a reasonable problem to cut your teeth on. I know you want to work this problem out yourself, which is admirable. Just remember, that you’ll learn the most by reading other people’s code rather than discovering the techniques yourself. So go ahead and let it get spoiled.

16

There aren’t many cases in which having the ability to do what the OP was specifically asking about strikes me as proper, but I certainly didn’t mean to imply that Frylock was wrong to even consider the idea. In fact, I quite vividly recall, when I was first learning to program, I had the same idea, and felt frustration at my inability to do such things in QBASIC. It’s just that, with experience, one perhaps can understand why such features could be considered problematic (the destruction of the soundness of alpha-conversion (the ability to systematically rename variables throughout code without changing its behavior, as ultrafilter mentioned) can make programs harder to reason about and more brittle/difficult to maintain). But that doesn’t mean there was anything wrong with originally being drawn to the idea in the OP; indeed, it was only a little tweaking away from the kinds of solutions I think most here would advocate, that tweaking being just such as to disengage it from the concerns raised by its original form.

I should say, although experience can breed wisdom, of course, there is also the flip side, the danger that the experienced programmer, having grown too familiar with the orthodox, begins to conflate the concept of “That which I have seen before” with “That which is possible”, or to take it to automatically set the limits of “That which is desirable”. Even if it had turned out that neither Python nor any other well-known language had features like the OP wanted, it would be perfectly plausible to still imagine a language design which did, and to discuss its merits and flaws. Something I often find quite refreshing in discussions with new programmers is their openness to all kinds of ideas on how to do things, and creativity in coming up with their own, whereas more seasoned hands can sometimes throw up more of a mental block, saying essentially “Well, that’s just not how it’s done”.

I see the same dynamic in play distressingly often with mathematicians/logicians, as well, but that’s a whole other discussion.

17

Here’s how I would do the volcano thing:

[spoiler]Have one class that represents a volcano. In this program it doesn’t do anything except store ingredients and remember whether it has exploded. Have one class that represents the store of available ingredients with methods to get some of an ingredient and to report what’s left. Have a manager class that acts as the interface and allows you to add some amount of each ingredient to a given volcano.

If you want the reaction to depend on the properties of the ingredients, there are several different ways. Google on decision tables for something that might be interesting.[/spoiler]

18

Okay, I now have something that works.

If anyone here would like to take a look at what I’ve written as a sort of “proof of concept”, and comment as to whether I’ve picked an inefficient or otherwise unnecessarily amateurish way to do it, (or else whether I’ve figured out just exactly how the pros do it :stuck_out_tongue: )I’d appreciate it.

Thanks for the conversation on this thread, as well. It’s been illuminating.

-FrL-

The “proof of concept” code is as follows. In execution, it is not user friendly, (if the user does something “wrong” the program doesn’t try to catch this and may act funny,) but the code is simple enough I assume you guys can follow it. Basically, it allows the user to create objects called “Beasts,” allows the user to feed any Beast he has created by specifying the name of the Beast he gave it, and allows the user to sell (i.e. delete) a named Beast as well.

The trick was to use a “bestiary” dictionary to associate user-defined names with actual instances of Beast.




bestiary = {}

class Beast:
    def __init__(self, name):
        self.name = name
    def feed(self):
        print self.name, "ate some food. Yum." #not necessary for this illustration, just did this first time around to be sure I could single out a single Beast by user-defined name
    def __del__(self):
        print self.name, "has been removed from your inventory."
        
def menu():
    choice = ""
    while choice != "q":
        print "Please choose from one of the following options."
        print
        print "(m)ake a beast"
        print "(f)eed a beast"
        print "(s)ell a beast"
        print
        choice = raw_input("?>")
        if choice == "m":
            print "What do you want to name this beast?"
            name = raw_input("?>")
            bestiary[name] = Beast(name)
            print
            print name, "created!"
        elif choice == "f":
            print "Which beast do you want to feed?"
            name = raw_input("?>")
            for n in bestiary.keys():
                if name == n:
                    bestiary[name].feed()
        elif choice == "s":
            print "Which beast do you want to sell?"
            name = raw_input("?>")
            for n in bestiary.keys():
                if name == n:
                    del bestiary[name] #It turns out this deletes both the Beast and the dictionary entry for it. I think.
        else:
            if choice != "q":
                print "That was not one of the selections listed."
    print "Bye!"

menu()
19

Here’s a slight modification to one of your bits that I think is worth looking at:



            print "Which beast do you want to feed?"
            name = raw_input("?>")
            if name in bestiary:
                bestiary[name].feed()
            else:
                print name, "is not a current beast."

If you had a program with a very large dictionary, this would perform better than iterating over all the keys and matching against a possible key. I find it a little easier to understand as well.

20

Thanks!

If I had thought of the non-iterating method, I would have done the following before you showed me that:

if name in bestiary.keys():
etc etc

because I didn’t realize I could just ask if it was in bestiary itself without explicit reference to its “keys”. So thanks for illustrating that fact for me.

I assume that “name in bestiary” will return “true” if name is either one of the keys or one of the values, right? (eta: Just did a test, and the answer is “no.” “name in dictionary” returns “true” only if name is one of the keys in the dictionary.)

-FrL-

btw Is there something you can do with a dictionary that you can’t do (albeit with more difficulty) with a simple and carefully formatted list?)