struts-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Frank W. Zammetti" <>
Subject RE: [OT] Re: environment awareness (project stage in JSF)
Date Sun, 29 Jun 2008 13:19:03 GMT
"In order to complete the test one the databases of one 
application was dumped to disk and moved to another available database 
server, and that individual applications configuration was updated to 
point at the new server (the other apps on the app server remained 
pointing at the original cluster). If the techies at DR just had a "dev, 
test, or prod" switch how could they have done that?"

I don't see how this says the environment switch doesn't work... This type of config is at
the app server level and can be changed independently for any given app regardless of what
env it's in... We do this sort of thing all the time too because of multiple versions in flight
and under QA review, but we still maintain a single EAR being deployed to all environments
for any given version.

-----Original Message-----
From: Al Sutton <>
Sent: Sunday, June 29, 2008 3:37 AM
To: Struts Developers List <>
Subject: [OT] Re: environment awareness (project stage in JSF)


It's always good to get a discussion going :).

The application isn't changed between boundries, it's the configuration 
that changes, and the configuration is controlled at each stage by the 
person responsible for that environment and not by the developer.

Let me give you a concrete example; During a DR test for a major bank 
the DR database cluster was found not to be able to handle the load of a 
full DR scenario. In order to complete the test one the databases of one 
application was dumped to disk and moved to another available database 
server, and that individual applications configuration was updated to 
point at the new server (the other apps on the app server remained 
pointing at the original cluster). If the techies at DR just had a "dev, 
test, or prod" switch how could they have done that?

Yes configuration is a feature of the environment, but it should be up 
to the environment manager to manage the configuration, and not up to 
the developer to dictate what is correct.


Brian Pontarelli wrote:
> Wow. This got way off topic, but I'm compelled to answer. ;)
> Your assumption about QA being the only problem is incorrect. It is 
> environment based. The application should not be changed in anyway 
> when it is migrated across boundaries. It should be able to determine 
> where it is and how it should behave. If you look at the differences 
> between environments, you'll see that this is true for many things.
> Your example of LDAP appears to support my case. You are providing an 
> environment based configuration source. In addition, your LDAP is 
> configured per environment via an IP constraint. Lastly, you've just 
> moved the configuration from files to LDAP. It is still different for 
> different environments. This could have been as easily done using 
> JNDI. What you are talking about IS environment awareness.
> In terms of payment processing, to be specific, we hit the gateway 
> using a different message that indicates it is a test. How do you 
> propose to determine if we should use a test message? The only way is 
> via configuration and that configuration must be different between 
> environments. However, you could as easily mock things out. But you 
> still have to contact a different URL. This is all based on environment.
> Again, I think you are simplifying this and by saying that people are 
> lazy. Trust me when I say that writing environment aware configuration 
> is not a reduction of work or management. Furthermore, you assume only 
> two environments it seems. We had 5 different environments and they 
> all had different configuration.
> -bp
> On Jun 28, 2008, at 10:06 AM, Al Sutton wrote:
>> Brian,
>> From what I can see your only real problem is QA on config files and 
>> given that how can you you can guarentee that all of your servers 
>> will never have their config drifted between zones because a certain 
>> problem occurr in dev but does in production.
>> I've previously worked on a project that used LDAP directories for 
>> everything (data storage and configuration). The app servers were 
>> only given the LDAP FQDN to bind to and pulled all of their config 
>> data from there. The LDAP servers had IP access control rules which 
>> prevented any machine outside of the domain attaching to them, this 
>> meant a server on the dev network couldn't get the production 
>> configuration and vice versa.  You could use an HTTP URL and web 
>> server as an alternative, but the principal is the same, protect the 
>> data which can cause things to go wrong (i.e. the config file), and 
>> don't try to code to prevent every screw-up a support techie will 
>> make (they can be pretty inventive when it comes to how to screw 
>> things up).
>> I can also see concerns over where do you draw the line between 
>> environments. With your example of credit card processing where would 
>> you say dev and production separate, do you write the code to return 
>> dummy auths and/or declines in dev mode, or do you call out to the 
>> payment gateway? One means that anyone with a spare machine can test 
>> something, the other means you need them to have the correct config 
>> and equipment to talk to the payment gateway?, what happens if 
>> someone wants to switch between the two in order to test the gateway 
>> interface, do you create another environment label?
>> All in all it does seem like a lazy solution to me, whats needed is 
>> better QA, not a solution which makes people sloppy because they 
>> think that the code will catch their mistakes.
>> Al.
>> Brian Pontarelli wrote:
>>> I think this is an over-simplification of a complex problem. Here 
>>> are a few examples from
>>> - Thread pool sizes. We couldn't replicate production (1500+ 
>>> servers) in staging, so instead, we created as many VMs as we could 
>>> handle on the limited number of machines we had (~100) to get an 
>>> accurate simulation. This required smaller thread pools to not kill 
>>> the OS
>>> - Different back end host connections to the GDSs. You can't book a 
>>> real flight in staging or development.
>>> - Different server names. We had around 7 tiers that spanned 
>>> multiple servers. Each request to Orbitz hits anywhere from 10-20 
>>> different machines. Although we used Jini to discover the services, 
>>> we still had to configure the Jini lookup servers differently 
>>> between environments
>>> - A classic example that everyone uses is database configuration and 
>>> SMTP servers. These are could be in a JNDI entry or the application 
>>> might create connections directly, depends. If the application 
>>> creates this stuff it will need different configuration per 
>>> environment.
>>> - Not charging credit cards in development, but charging them in 
>>> staging and production. And we also had specific merchant accounts 
>>> to test in staging that were full transactions, but they didn't 
>>> charge us the full amount. We also had many different bank accounts 
>>> setup to test all the different types of cards and transaction 
>>> boundaries.
>>> And the list continues. I might agree that an MVC might not need to 
>>> know the environment, but an application will. The example you give 
>>> with logging has very little to do with environment concerns and 
>>> more to do with poor testing and programing. In addition, you should 
>>> have been able to turn it off.
>>> I think a better example of bad environment configuration is using 
>>> it to configure everything and having complex and error prone 
>>> configuration files. I recall two cases that are quite humorous:
>>> 1. With Jini we could dynamically add machines and the system would 
>>> discover them and they would immediately start accepting work. Made 
>>> scaling simple. Someone had setup a box and mistakenly named the 
>>> environment to "pr0d" (yeah that's a zero in there). Took us hours 
>>> to figure that gem out and at 2am no less.
>>> 2. Someone was creating a new service to interact with a new GDS 
>>> feature that provided discounts on hotel rooms. They were testing it 
>>> out in development and being a developer, thought a 98% discount 
>>> would be some good test values. Rather than putting the value in the 
>>> file it ended up in the 
>>> file and made it all the way out to 
>>> production. The hotel called us up and mentioned that they had 
>>> quickly sold out over New Years at a whopping $6 a night. Luckily 
>>> they only had 5 rooms or something, but we ate the cost of selling a 
>>> 5-star hotel at 98% off.
>>> I think the principle is sound, just needs a lot of testing and 
>>> understanding. I definitely don't think it has anything to do with 
>>> lazy developers. In fact, some of the best developers I know use it 
>>> extremely well to control size, performance, scale, functionality, 
>>> and much more in different environments.
>>> -bp
>>> On Jun 28, 2008, at 4:56 AM, Al Sutton wrote:
>>>> I think the concept is an idea which will appeal to lazy developers.
>>>> Why on earth would you want to put conditionals into your code that 
>>>> you know will only evaluate to a set value in the environment they 
>>>> run in?
>>>> If anything it makes problems harder to track down because if 
>>>> someone takes a copy of the app from a production machine to a dev 
>>>> machine to further investigate a problem it will behave 
>>>> differently, which is just a hiding to nowhere in multi-threaded 
>>>> apps such as S2 webapps.
>>>> An example of one of the "joys" that can come from this type of 
>>>> idea came from a project I worked on where a coder used log4j and 
>>>> isDebug to conditionally build a log string and log some extra 
>>>> data. This might be seen as a good idea, except the code within the 
>>>> conditional block didn't properly check all the objects were not 
>>>> null and under certain functionally valid conditions an NPE was 
>>>> thrown, so when a problem arose in production at a customers site 
>>>> they were asked to turn debug logging on and all that they sent 
>>>> back was a log with an number of NPEs which didn't relate to the 
>>>> original problem.
>>>> Ohhh the fun we had explaining that a new release had to go through 
>>>> their change (long) control procedure just so we could find out 
>>>> what the original problem was and until that we we're kind of 
>>>> stuffed finding out what in their environment triggered the problem.
>>>> Yes in an ideal world it shouldn't have happened. Yes it probably 
>>>> should have been picked up by some QA test somewhere. But don't we 
>>>> all live in the real world?
>>>> Al.
>>>> Chris Pratt wrote:
>>>>> We use something similar in our system.  The system uses a bunch of
>>>>> resource bundles that are separated into logical domains, and each
>>>>> entry can be overridden by a local file on each machine.  Plus each
>>>>> entry can be scoped by environment (production, test, development),
>>>>> machine, or application name (in case multiple applications are
>>>>> sharing a library component).  We have log4j and spring 
>>>>> configurers so
>>>>> that it is tightly integrated into the tools we use.  It's saved 
>>>>> us an
>>>>> eternity of time tracking down bugs from one environment to the next
>>>>> since we deploy the same WAR file that was accepted by the quality
>>>>> assurance group into production and let the configuration take 
>>>>> care of
>>>>> itself.
>>>>> I've often thought of creating a Google Code project to open source
>>>>> it, but wasn't sure if there would be enough interest.
>>>>> (*Chris*)
>>>>> On Fri, Jun 27, 2008 at 1:38 PM, Brian Pontarelli 
>>>>> <> wrote:
>>>>>> Yeah, I found that environment resolution was a key feature for 
>>>>>> any large
>>>>>> application. At Orbitz we could deploy the same bundle to any 
>>>>>> server and the
>>>>>> bundle would figure out where it was and configure itself for that
>>>>>> environment. Worked really well.
>>>>>> I have also provided this type of feature in JCatapult using an 
>>>>>> API that can
>>>>>> be implemented however developers need. The default 
>>>>>> implementation uses
>>>>>> JNDI, but it is simple to change it. The nice thing about that is

>>>>>> you can
>>>>>> assume at all times that the environment is available and make 
>>>>>> assumptions
>>>>>> around that.
>>>>>> -bp
>>>>>> On Jun 27, 2008, at 1:53 PM, Frank W. Zammetti wrote:
>>>>>>> We d something similar as well, but we decided to use a simple

>>>>>>> env var in
>>>>>>> all environments... So the exact same EAR can deploy to any 
>>>>>>> environment and
>>>>>>> the code within simply looks for that var and acts accordingly.
>>>>>>> Simple but
>>>>>>> highly effective.
>>>>>>> Frank
>>>>>>> -----Original Message-----
>>>>>>> From: Ian Roughley <>
>>>>>>> Sent: Friday, June 27, 2008 2:59 PM
>>>>>>> To: Struts Developers List <>
>>>>>>> Subject: Re: environment awareness (project stage in JSF)
>>>>>>> I've actually had to implement this type of feature in multiple
>>>>>>> enterprise applications.  However, I would say that it's not

>>>>>>> knowing the
>>>>>>> environment, but being able to change configuration elements
>>>>>>> environment that is important (for what I did, and in rails I

[The entire original message is not included]

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message