Thread: Ask me about Python

Okay this is just a simple question, but how does Python differ from C and C++? Because I'm trying decide on a language.

Python is an easier language to start with so you should probably start with it as opposed to C or C++. From a programmers perspective, the differences are that

1. python has a much larger standard library and so will take care of many common tasks for you.
2 It has built in garbage collection so that you can dynamically create new variables and not have to worry about releasing the memory


The downside is that it will not be as fast (although I have seen good python code outperform mediocre C for computationally intensive tasks - it was using highly tuned C functions to do it though For most cases, this difference in speed doesn't matter.

Python also allows functional, object oriented and procedural programming styles so it will give you a lot to play around with.

It also integrates very well with C and C++ so that when you learn those languages, you can still write in python and profile your code to find the bottlenecks and re-implement only the necessary parts in C.

I haven't read (python is not my first language) but if you are just starting out in programming, I have heard very good things about Think Python.

According to this site, it was Assur. Howzit?

Is it possible to shut off the garbage collector? Professionals hate garbage collectors.

C++

Code:

struct autolock {
   autolock(pthread_mutex_t * lock){ mylock = lock; pthread_mutex_lock(lock); }
   ~autolock(void){ pthread_mutex_unlock(mylock); }
   private : pthread_mutex_t * mylock;
};

{   //critical section
   autolock lock(mutex);  //never have to explicitly unlock the mutex
   if (something) return;
   if (something) return;
}  //lock goes out of scope and is immediately deleted

If there is a garbage collector

Code:

{   //critical section
   pthread_mutex_lock(mutex);  //have to unlock the mutex explicitly for EVERY path out of the critical section
   if (something){
      pthread_mutex_unlock(mutex); 
      return;
   }
   if (something){
      pthread_mutex_unlock(mutex); 
      return;
   }
   pthread_mutex_unlock(mutex);
}

See why pros don't use languages that have garbage collector?

And even though it looks like the C++ has to do more, creating the objects and such, because the timing of the object destruction is defined, the compiler can optimize the hell out of it and it will run faster than the garbage collected code.

I need to write a python program soon which generates simple musical sequences (monophonic) in real-time according to various parameters. Tone quality is not essential but reliability and efficiency are (and ease of use is a bonus). Do you have any recommendations on libraries etc.?

Originally Posted by ninja9578

Is it possible to shut off the garbage collector? Professionals hate garbage collectors.

Professionals don't hate garbage collectors. Professionals like to get as much work done with as little code as possible in the most readable manner because that allows for a better program. That's why professionals use python when it's appropriate.

Anyway, your example is garbage. You are referring to a limitation of the C library pthreads which has nothing to do with either garbage collection or threading in general.
Python

Code:

import threading

mutex = threading.lock()

# critical section
with mutex:
    if something:
        return something
    elif something_else:
       return something_else

# mutex is automatically released when we exit the with block through any path 
# this includes uncaught exceptions

See why professionals like python where it's appropriate?

When you started talking about garbage collectors, I thought you were going to bring up the usual (and valid) criticism that they can pause program execution when they are running. so you would not want a garbage collector in a pacemaker or other time critical piece of software. Given the fact that most large C/C++ programs for which this is not a concern end up manually reinventing a garbage collector anyways (or using a third party one, or leaking like the titanic), i think that it's just that much more ridiculous for you to claim that 'professionals' don't like garbage collectors and reflects a sort of macho, speed is everything attitude that I often see from C/C++ programmers.

I think it's because you're mad that a long running java (ugh!) process can outperform a long running C++ process due to microoptimizations that the JITC can make based on runtime conditions that aren't available to even the smartest static compilers.

Originally Posted by DuB

I need to write a python program soon which generates simple musical sequences (monophonic) in real-time according to various parameters. Tone quality is not essential but reliability and efficiency are (and ease of use is a bonus). Do you have any recommendations on libraries etc.?

I haven't done any audio programming but it looks like pyo is your best bet. It's written in C so it should be very low overhead to call into. If you write correct python code, then efficiency and reliability shouldn't be a problem. I can surely help with that but It's hard to say more without knowing what your parameters are and how they map to output.

Where'd my damn post go?

Originally Posted by Xei

Where'd my damn post go?

Seriously. You've been around long enough so that you should be able to spam all you want. Can whoever deleted xei`s post delete my response to it as well? it breaks continuity.

Originally Posted by PhilosopherStoned

Anyway, your example is garbage. You are referring to a limitation of the C library pthreads which has nothing to do with either garbage collection or threading in general.

No I wasn't; I was demonstrating how C++ programmers use the predictable behavior of when destructors get called. If there is garbage collection, you have no idea when destructors get called, so you can't use them the way you can in C++. pthreads's locking mechanism was just to show how scope is used in C++, I could have also show glBegin/glEnd statements paired together, or new/delete (called scoped pointers.) When things go out of scope' they are immediately destroyed, so you can put code in your destructor to do work. With a garbage collector, you can't do work, you can only clean the object up. Maybe python does it differently, but thats how it works in java and PHP.

BTW, java will never be faster than native code, that's only possible on the SPARC processor.

Scripting languages should stay on the web because they're portable, real apps will always be written in native compiled code, without garbage collection. The only "real" language that I know that has a garbage collector is C# and Objective C 2.0, and professionals turn it off in both languages.

Originally Posted by ninja9578

No I wasn't; I was demonstrating how C++ programmers use the predictable behavior of when destructors get called. If there is garbage collection, you have no idea when destructors get called, so you can't use them the way you can in C++.

What I don't get is why C++ programmers always insist that if you can't accomplish something the way that you do in C++ then it's not as good. I've literally had someone try to tell me that C++ is more powerful than python because python doesn't have a 'do' loop. One would often use the 'with' statement to have a context manager have code run when exiting a scope as I demonstrated in my example (though it's much more flexible than that)

Originally Posted by ninja9578

BTW, java will never be faster than native code, that's only possible on the SPARC processor.

Keep telling yourself that. Optimizing code at compile time as pretty much been optimized to it's fullest extent. Optimizing code at runtime using Just In Time compilers has only just started and is theoretically much more promising. And note that I specified long running processes so that the JIT compiler can accumulate enough information to, e.g. do branch prediction superior to what's currently available in hardware.

Originally Posted by ninja9578

Scripting languages should stay on the web because they're portable, real apps will always be written in native compiled code, without garbage collection. The only "real" language that I know that has a garbage collector is C# and Objective C 2.0, and professionals turn it off in both languages.

You really seem to be hung up on destructors here if you are willing to dismiss an entire class of very expressive and powerful languages for no reason other than that they make destructors useless. I recognize that clever things can be done with them but clever(er) things can also be done with context managers, closures, descriptors, decorators and metaclasses, none of which are offered in C++.

I'll repeat myself.

favor C/C++ when you need the speed (and after you know that another language isn't fast enough)
favor C/C++ when pauses that are (increasingly only theoretically) associated with (modern) garbage collectors is a game killer.
favor C when you are writing for an embedded system

in all other cases

use a modern language, e.g. python, lua, ruby and most of all, Lisp.

I guess what I'm saying is that (while I may be 5-10 years ahead of schedule) C and C++ are the new assembler languages.

Originally Posted by PhilosopherStoned

Seriously. You've been around long enough so that you should be able to spam all you want. Can whoever deleted xei`s post delete my response to it as well? it breaks continuity.

It was a joke. Monty Python? Ask a question? Capital of Assyria? :l

Originally Posted by Xei

It was a joke. Monty Python? Ask a question? Capital of Assyria? :l

Hey, I said I knew python I know it's named after Monty Python but I'm not much of TV watcher so a lot of the references go over my head. I knew it was a joke though at least. as was lmgtfy.

I don't think jokes should be gratuitously deleted by the forum Gestapo.

Originally Posted by PhilosopherStoned

I haven't done any audio programming but it looks like pyo is your best bet. It's written in C so it should be very low overhead to call into. If you write correct python code, then efficiency and reliability shouldn't be a problem. I can surely help with that but It's hard to say more without knowing what your parameters are and how they map to output.

Thanks, I'll look into pyo.

To answer your question about the parameters I'm planning on using, I recently wrote a very brief paper (~1500 words) for some of the faculty in my department which summarizes the experiment I'll be using the program for. I think it offers enough detail to give a sufficient picture of the kind of requirements I'm looking at. As a one-line summary, it is meant to investigate a potential feature of musical melodies which causes people to judge them as interesting and/or pleasing. That's right, we're doing science on music now, bitches; nothing is sacred. Anyway, I've uploaded a PDF of the paper here, to a site called filedropper.com, which I've never used before. Let me know if you have any problems accessing the paper.

What does it do? And how can I benefit from such a subject?

Originally Posted by Jorge

What does it do? And how can I benefit from such a subject?

it's a programming language. You can make programs.

Originally Posted by DuB

To answer your question about the parameters I'm planning on using, I recently wrote a very brief paper (~1500 words) for some of the faculty in my department which summarizes the experiment I'll be using the program for.

Sounds fascinating. Do you know how to generate the coefficients so that the resulting trig sum generate pink noise? That's not an area of math I've done too much with (though it would be fun to get into). If you can do that, then

Code:

import math

def make_wave(coefficients):
    def wave(x):
        return sum(c * math.cos(i * x) for i, c in enumerate(coefficients))
    return wave

Is an example of how to put them together. This creates a function make_wave that returns the actual wave function. to use it, just pass it the coefficients in a list or tuple and it will return a function that takes a single numeric variable and returns a float representing the value of the trig sum on that number.

Code:

durations = make_wave((1, 2, 3, 1.4, 12, .001))

print durations(1)
print durations(2)
print durations(1.4)

But you want to partition the image of the waves into discrete pieces for duration and note. There's two ways to figure that out. you can either have a fixed set of intervals that applies to all functions and only generate functions whose output falls within the range of the function or you can determine the range of each function and partition it equally. From your paper, I'm not sure which you intend to do (I might have missed it). Again, I'm not up on my signal theory. Is equal length partitions what you want or should their lengths be weighted?


Their might be a library to do a lot more than just generate sounds as well. I'll poke around.

Originally Posted by PhilosopherStoned

in all other cases use a modern language, e.g. python, lua, ruby and most of all, Lisp.

I'm not sure I'd say lisp is a "modern" language... It's the second oldest high-level language, originally specified in the late 50s.

However, you mention that you learned Python after other languages. Do you have any good tips for getting into the python mindset? When I try scripting in it, I find myself googleing "what's this PHP/C++ function in Python" a lot :p

Originally Posted by khh

I'm not sure I'd say lisp is a "modern" language... It's the second oldest high-level language, originally specified in the late 50s.

That doesn't mean that it's not the most modern, it just means it was (and still is) remarkably ahead of its time. It's the most powerful language I could possibly imagine.

Originally Posted by khh

However, you mention that you learned Python after other languages. Do you have any good tips for getting into the python mindset? When I try scripting in it, I find myself googleing "what's this PHP/C++ function in Python" a lot :p

That's sort of a general question and it would be best if you tossed up some code or at least the specifications for a reasonably simple program. Are there any particular things that you find yourself wanting to do a lot? I might be able to suggest some functions to get to know better. enumerate, zip and the whole itertools module come to mind as good things to be familiar with.

One guideline is that you should not have much (if any) boilerplate code. You should solve each problem only once. For example, suppose that you're writing a function to processes some item. Sometimes you want to process only one and sometimes you want to process a (non-dictionary) collection of them and build a list of the results. Here's the curly brace way of doing it in python

Code:

    def widgetize_foo(foo):
       return Widget(foo.a, foo.b, foo.c)

    foo_widget1 = wigetize_foo(foo1)         # process just one

    foo_widgets = []  
    for foo in list_of_foo:        # process a list
       foo_widgets.append(widgetize_foo(foo))

There's at least two problems with this.
1) You are going to be rewriting that same for loop everytime you want to process a list of foos.
2) If you later add a new way to make a widget from foos, you have to create a new function to do it and then write more for loops every time you want to create a list of the new widgets.

Another problem with it is that that's the slowest way to build a list but people that don't know better often use it. Use it only if you can't do it with a reasonable list comprehension.

The correct way to do this is to use a decorator.

Code:

def with_many(f):
    def process_many(collection):          # collection can be any sequence or a set
        return [f(item) for item in collection]          # use a list comprehension
    f.many = process_many  
    return f

Now define the function to process one foo and apply the decorator to it.

Code:

@with_many
def widgetize_foo(foo):
    return Widget(foo.a, foo.b, foo.c)

foo_widget1 = widgetize_foo(foo1)      # process just one like normal
foo_widgets = widgetize_foo.many(foo1)     # process a whole list of them

So this is slightly more lines of code for only one list but as soon as you do it a few times, you're reducing code repetition and, because the decorator is just a higher order function, it can apply to any other function as well. So now when you add a second type of widget that you can get from foo, you can just apply the decorator to that factory function and not have to write loops for it either. Essentially, you write one loop (or list comprehension) and cut out a whole class of code.

Now granted, in C++ or PHP, you could abstract this processing away as a function and you could take advantage of either templates and loose typing [] in C++ or loose typing in PHP but doing it in python, you get a simple way to apply a strongly typed solution to a broad class of problems using a very expressive notation.

C++ is not loosely typed, it's one of the strongest typed languages there is, behind only Ada. And you can turn on compiler options to make it just as strong as Ada. Polymorphism is not the same as loose typing. And just because C-casts and opaque pointers exist, doesn't mean they are used.

Originally Posted by ninja9578

C++ is not loosely typed, it's one of the strongest typed languages there is, behind only Ada. And you can turn on compiler options to make it just as strong as Ada. Polymorphism is not the same as loose typing. And just because C-casts and opaque pointers exist, doesn't mean they are used.

Of course polymorphism isn't loose typing. Polymorphism is one of the foundations of good design. void* is loose typing and I see it used all over the place (at least in C). "Exists in the language" means the language is not strongly typed. You may program in a subset of it that is (and I commend you for that) but you can't seriously claim that a language that lets you treat any type as any other type is strongly typed.

I was looking for some python help and hopefully you're still able and willing to answer some questions here, so I'll just post them and hope for the best

I'm working on an IRC proxy (or bouncer) written in Python 3. In order to make certain things easier I've extended the socket class.

Code:

class IrcSocket(socket.socket):
	def __init__(self, id):
		self.id = id
		socket.socket.__init__(self, socket.AF_INET, socket.SOCK_STREAM)
	def send(self, bytes):
		log(bytes)
		socket.socket.sendall(self, bytes)
	# etc (that's just a dummy)

However the server socket is a normal socket, and the socket.accept() function returns a normal socket... how can I turn this into an IrcSocket?

!!!
edit: Ignore the following question. I read the fucking manual.
!!!

Since it's mainly running on windows, and is intended to only have a handful of connections, I'm using one thread for each connection. I'm using a dictionary of normal queues to pass messages between threads.

Code:

messages = {}

class IrcServer:
	# __init__, etc
	def handleMessage(self):
		(type, data) = messages[self.id].get()
		if type == 'ircChildren':
			for (k, q) in enumerate(messages):
				if k > self.id:
					q.put(('irc', data))
	# Other stuff

class IrcServerChild:
	def __init__(self, id):
		self.id = id
		messages[id] = queue.Queue(0)
	# Other stuff

However this fails with the error message

Code:

Exception in thread Thread-2:
Traceback (most recent call last)
  File "C:\Python\lib\threading.py", line 736, in _bootstrap_inner
    self.run()
  File "C:\Users\khh\Documents\khh\python\kircp\src\ircserver.py", line 113, in run
    IrcServer.handleMessage(self)
  File "C:\Users\khh\Documents\khh\python\kircp\src\ircserver.py", line 141, in handleMessage
    q.put(('irc', data))
AttributeError: 'int' object has no attribute 'put'

I don't get why q is an 'int' and not a 'Queue'.

It's kinda tough for me to help with this as I'm limited to coding in javascript at the moment. Hence I can't post running code.

It seems like it would be best to make IrcSocket a wrapper class that provides the necessary functionality using parasitic inheritance instead of classical inheritance. IIRC, socket.socket is a c class and things can get pretty hair extending those. Something like

Code:

class IrcSocket:   #No need to extend object because this is p3k
    def __init__(self, id, socket):
        self.id = id
        self._socket = socket
 
    def send(self, bytes):
        log(bytes)
        socket.socket.sendall(self._socket, bytes)

Then you can get your hands on one with something like

Code:

 
  ircsocket = IrcSocket(some_id, socket.socket(socket.AF_INET, socket.SOCK_STREAM))

This just instantiates the socket inline and immiediately passes it to the IrcSocket class constructor.

You can just do

Code:

 
wrapped_socket =  IrcSocket(some_id, socket.accept(*args))

to wrap the regular socket that socket.accept returns. Again, you're just making the call inline and passing it to the constructor.

Like I said, that's psuedocode because I can't run it and I don't have the library memorized but that's the basic idea.

Oh yeah. I just remembered that you'll want to override __getattr__ for that to work. And it occurs to me that socket.accept is a method on socket.socket and not a module level function of socket as I treated it in my first post. So the code will be something like:

Code:

class IrcSocket:   #No need to extend object because this is p3k
    def __init__(self, id, socket):
        self.id = id
        self._socket = socket
 
    def send(self, bytes):
        log(bytes)
        socket.socket.sendall(self._socket, bytes)

    def accept(self, *args, **kwargs):
        tmp_socket = self._socket.accept(*args, **kwargs)
        return type(self)(some_id*args, tmp_socket)

   def __getattr__(self, attr):
       return self._socket.__dict__[attr]

The type operator returns the type of the argument (when it's called with one argument. with three, it creates a new type/class). Here we're using it to get at IrcSocket. Doing it that way avoids hardcoding the class into the code so that derived classes will return derived classes rather than IrcSocket.

__getattr__ will fetch attributes that can't be found on IrcSocket from its _socket attribute.