Agree with cal97g - the thing is difficulty won't keep going up by 40 or 50 per day. Whilst we're where we're at now - with a lower difficulty and (let's say in relation to the future) low total MH/s.
In the future as more hashing power is added to the network, the difficulty will rise yes - but it's there to balance out the finding of blocks. those with low hashing power will drop off if its not worth it. This will bring the total hashing power of the network down, and difficulty will decrease. A natural balance will be found. Whilst there's still an absence of ASICs, expect GPU mining to be worth it (maybe not on very old cards and single GPUs, but multi card setups that are reasonable hashing) .
However one thing that will make a big difference is people jumping ship from BTC to LTC mining... Especially those with a large number of GPUs. That said, some will - some will sell their gear for a return - and I think others with a large amount of kit will "spread their bets" and split their kit between multiple currencies.... PPC and TRC are pretty low in value at the moment, and there is a temptation to get in their with a high hash rate for possible future greater returns... As well as putting some mining power into LTC as that is the most mature (and therefore most likely to succeed at this point) after BTC.