Thursday, 4 October 2012

aukema.org: Exchange Client Network Bandwidth Calculator

aukema.org: Exchange Client Network Bandwidth Calculator:

'via Blog this'

Original page copied without editing for my sanity. Sites seem to disappear without notice and that annoys me when I need to find something urgently. Go to the link above for the original.

----------------------------------------------------------


THURSDAY, MARCH 22, 2012

Exchange Client Network Bandwidth Calculator

Last night one of my friends in IBM asked my opinion about the Exchange Client Network Bandwidth Calculator (it's here). I gave him my response, after I had downloaded and played with it. Here it is.

Since 10 years, my work and passion is all about analyzing network traffic for large Lotus Domino customers, I have detailed network and usage statistics for more than a million Notes users in my DNA data warehouse.  And I maintain benchmarks to compare customers against. So I think I can judge.

From a first look and play with this 2012 calculator spreadsheet, it looks pretty complete in the sense that Microsoft have incorporated a lot of variables to represent end user behavior. Now let's see what happens if we run a couple of scenario's:

Scenario A: 10,000 users with a Medium Profile
  1. Leave all parameters on their initial default values in the worksheet named 'Input';
  2. In the worksheet named 'Client Mix', specify all users to have a Medium Profile at each site and in such a manner that every version and type of Outlook is assigned 10,000 users;
  3. Add a column to the right where you divide the resulting bandwidth by the number of users, thus calculating the kilobits per user: (=1024*Table47[@[Network Bandwidth )
Click to enlarge
Recommended network bandwidth is appr. 4 kilobits per second, per user.


Scenario B: 10,000 users with a Very Heavy Profile
  1. Again leave the parameters to their initial values;
  2. Change the Site User Profile to 'Very Heavy';
Click to enlarge
Recommended network bandwidth ranges from 6.5 to 11.75 kilobits per second, per user.

My observation:

Let's apply the Microsoft Recommendations to the network bandwidth requirements I reported to an organization with 9,500 users and 101 office locations world wide.
Click to enlarge
The blue bars are network demand levels observed over a 7 day period, as a consequence of measuring the real end user demand in those locations. The two red lines come from Microsoft's estimates for Medium and Very Heavy user profiles, that I applied to the user count in each of the office locations.

Of the 101 office locations, Microsoft's estimations would result in a severe shortage of network capacity for 6 locations, even when applying Microsoft's most heavy profile estimates.

Would we follow Microsoft's Calculator for Medium Profiles, more than 50% of all office locations would end up with severe shortage in their network capacity. For the remaining locations, the customer would end up with significant over capacity (56% to be exact, averaged across all sites).

Why is the Microsoft Calculator giving wrong estimates for almost every site?
Do the default user profiles not represent the behavior of users at my customer properly? Do the users in my customer send and receive more messages, and are these messages perhaps smaller or larger than what Microsoft put in their profiles? I don't think that matters. Putting more load per user profile would lead to even more over capacity at half of the sites, while still leading to shortage in others.

The real problem with this calculator is putting people into profiles. You cannot predict end user behavior, you can only observe what they are doing today. It's ok to classify your observations into profiles and categories to e.g. present pretty pictures. But it is very dangerous to use those classifications and generalize entire user populations for sizing purposes.

My Conclusion:

Predicting network requirements will almost always result in either over-capacity (waste of money) or under-capacity (end users complaining about poor performance).

With DNA, we measure the Real End User Demand over a relevant 7 day period. It is far safer to expect that end users will show the same behavior tomorrow, as we observe today, than to apply an assumption that 4 or so profiles fit all your end users.

My Recommendation:

If your sole objective is to migrate away from Domino, regardless the consequences and as quickly as possible, go ahead and use Microsoft's Network Bandwidth Calculator. You'll love it.

If you prefer to make well-informed, fact based decisions with regards to your platform strategy, based on the Real End User Demand, ask IBM to perform a detailed study on the end user demand in your organization.

Side Note:
I am not an expert in Microsoft Exchange or Outlook. Although theoretically it could be possible that network consumption of outlook users is significantly lower than a typical Notes user, I doubt that this is the case.

Sunday, 18 March 2012

I am still around. There is really nothing to report in Lotus land apart from shrinkage. Perhaps the ice caps have melted and the sea is rising but whatever has happened the estate is rapidly shrinking. Some will disagree and rightly so as pockets of resistance is still active. If you are a fan of Firefly then know is probably the time to look for an old junker to move freight around the planet. I think command central has disavowed the yellow bleeders. My opinion only.

That said, the platform is still awesome. I can still setup a working highly secure hugely scale able collaboration system second to none on virtually any platform.

If I had to start a resistance movement I would use Lotus Domino for sure. Viva revolution.

Perhaps I should start a YouTube series to show all just how amazing this product is. It will be difficult for me not to get cynical though but not because of the product or the developers or the support staff or the BP's but rather by the utter contempt the leadership has for the product of the decade.

It is a crying shame but nevertheless, I am now considering starting an advanced video series.

Hmmmm, he says pondering away.

Monday, 29 November 2010

I have a new hero - Mat Newman - Admin and Designer in Linux Wine

Coming back to Linux has been great and Ubuntu 10.10 (Maverick Meerkat) is a pleasure to use. The next stage is to get Lotus Notes and Domino running but as always there is a fly in the ointment. I remember a conversation with Jalan H 4 years ago asking her about the Quickr client for Linux and she said there is no demand for it but it is on the road map. I guess the same is true about Designer and Admin for Lotus Notes under Wine. It is on the road map.

Linux has a feature called WINE (WIne is Not an Emulator) that is a set of API's that understand what a Windows application call requires and is serviced with a thunking call or a Linux function. The upshot is that you can run a simple Windows application in Wine without virtualisation.

More complex programs require reverse engineering to work in Wine and there are whole teams dedicated to make it happen. The team working on making Lotus Notes 8.5.2 work on Wine is a team of one called Mat Newman!

Mat published a blog post called:  Quick ’n’ dirty - setting up Designer under Ubuntu

My Notes build was copied for the IBM Windows 7 build and unfortunately, in this case, it is a 64 bit Notes install. Please don't misunderstand me, Notes is not 64 bit but I guess some of the Eclipse bits are 64 bit so the message I get is:

Trying to load PE image for unsupported architecture (AMD-64)

Administrator works for now though! I will build a version of Notes on XP later today and test again.

The reason nobody tried this is that the last version that worked was 7.0.x. We could never get the Eclipse version installed as it always barfed. In steps Mat and thinks outside the inbox!

Three hoorays for Mat! Hip hip....

:-)

Sunday, 28 November 2010

Moving back to Linux

A while ago I posted that Linux has become a distraction. As I moved back to services it became clear that several documents and spreadsheets with macro's etc would not work in non Microsoft environments. Also, I spent quite a bit of time tweaking and playing. The major issue for me and all Lotus power users is no Designer support in Linux. That is a blessing in disguise though as Designer in Windows redefine the word slow and extenuate the word frustrating. Anyway, my buddy Mat Newan posted how to get Designer kind of working in Wine that was quickly followed up by my other buddy and fellow IBMer Dave Hay reminding me that Wine is against IBM security guidelines. IBM blue tape extends far and wide, believe me. However, the machine in question is a lab workstation ;-)

In fact what I have now realized is that Linux is a differentiator and in a doggy eat doggy world having unique skills is rather important so back on the band wagon I jump!

My first requirement though is my Windows 7 workstation in WMWare. I have no choice.  No Windows no go. Even in the company that openly speaks of great support for Opensource Windows is king.

My Thinkpad T400 has an Intel Due Core P8600 install so 64 bit hosts under 64 bit Ubuntu 10.10 was no issue. I used our Lifeboat process and created the final test build of Windows 7, yes we are about to enter the 21st century, and all was good. The next step was to convert my Windows 7 build to VMWare using VMWare converter. The first try did not work well at all. After some tweaking all was good and I am now running my Windows desktop in a VMWare window. The only issue so far is that the AT&T (most frustrating program ever) does not create the VPN tunnel.

On my home machine it was less fun. I am no Oppenheimer so my machines are old. I got VMWare installed and then go the dreaded your host does not support 64 bit message. Not fun at all. Here is how it got resolved.

First the realization that I have an issue. Bring on the iceberg syndrome.

http://rayslinux.blogspot.com/2010/11/vmware-64bit-chip-support.html

Then the possible workarounds for my new found issue.

http://rayslinux.blogspot.com/2010/11/cpu-vt-flag-for-64-bit-host-on-64-bit.html

And then finally after some frustrating moments the problem is history!

Running a 64 bit host in KVM or VMWare Linux - Intel E6500 chip
http://rayslinux.blogspot.com/2010/11/running-64-bit-host-in-kvm-or-vmware.html

A happy ending for now.

Next steps, Designer under Wine and then Domino 8.5.2.

Fun Fun Fun!




   

Sunday, 21 November 2010

On-Site Rules for Outstanding Consultants - Alan Weiss

Sometimes we need reminding of the following to ensure customers are satisfied. It is not always easy as we do not live in a perfect world. When you are on a technical engagement it is even more difficult to summarize at the end of the day especially when things did not go according to plan. You try and diagnose why STLinks do not work in a network with multiple proxies, firewalls and packet shapers on the internal network and then summarize for the client. However, if you do not do what Alan says your hard work may just turn to dust right in front of your eyes.


From Alan's Blog:
When you’re with a client, before you depart, try to:

• Summarize progress and current status.

• Have client agree with positive results to date.

• Commit to his or her accountabilities in near-term.

• Agree on time and date for next discussion between you.

• Secure follow-up on any internal issues that need action/correction.

• Acquire referrals (if you’re more than half-way through the project).

• Provide value about non-project and peripheral issues.

• Personally see all key stakeholders who are present.

• Find out if anyone is waiting for anything from you.

• Seek feedback on any new initiatives you’re considering (e.g., new product or teleconference).

• Thank any assistants or secretaries who have been of help.

• Observe the environment for any changes.

• Raise status on any overdue fee or expense reimbursement payments.

• Learn of any key changes in the company’s condition (e.g., earnings, attrition, technology, etc.).

• Be seen by as many people as possible, including potential buyers.

Thursday, 18 November 2010

IBM i and Enterprise upgrades - Ensuring MSSO works.

Another 10 minute job that took three days, well not quite but it felt like it. We upgraded the dev environment for a big project I am running to 8.5.2 and the next step was to test SSO between Portal 5.1 and Domino. We accessed the Portal environment and clicked on the email icon. The next screen we expected was iNotes 7 mail. What we got was enter user credentials. #fail!

For those of us with limited attention spans, or certainly most twitter users, I have documented the answer here with my experiences following. The issue was to make sure that the domain field is properly populated in the LTPA setup in Portal. If not, Portal will issue a token using a hostname and not domain. Domino will not use the hostname for authentication as it requires a wild card domain for SSO authentication.
Of course it was immediately assumed that Domino was to blame as it was the last services to be upgraded. #fail 1.

Well, as with all challenges more is learned when things go wrong so here are my experiences.

The first thing I spotted in the Domino directory was the Web SSO Configuration document. It was set to Token Format:- LtpaToken (compatible with Domino 7 and prior releases) I immediately made an assumption that this is could be the issue (shame on me). I changed this field to LtpaToken and LtpaToken2 compatible with all releases. #fail 2.

Firstly, duh, Portal is authenticating and passing the token to Domino so this field is irrelevant. I can save you asking uncle Google now and let you know that all this field does is issue two Ltpa tokens using one key when authenticating to Domino.

This is a good document for some background information.

Here are the convoluted steps I took because I did not focus and made assumptions.

Activate debugs on Domino to see what was going wrong.
tell http debug session on
tell http debug thread on
tell http debug postdata on
tell http debug responsedata on
set config WEBAUTH_VERBOSE_TRACE=1
set config WEBSESS_VERBOSE_TRACE=1
set config LDAPDEBUG=7
set config debug_sso_trace_level=3
stop consolelog
/data/IBM_TECHNICAL_SUPPORT/console.log>
start consolelog

Top tip: Add the commands to custom commands when using the console in Domino Administrator.

These debugs provide wonderful reams of information that shows you what Domino is up to. However, you will soon be overwhelmed and crave the simplicity of no debugs! There was nothing really in the debug. I could see the LDAP server find the name and resolve properly and all seemed ok.

Next step was to go back to the SSO docs and check the realms and slashes. One confusing point was that the realm should add on a \:389 but due to the magic of Domino 7 upwards you do not need to add the \ before :389. Confusing but helpful.

Now for the "quirkiness" of Domino. Rant removed for own sanity.

Domino HTTP can operate in two distinct ways. It can use a legacy configuration to make sure it is compatible with older methods and to ensure migrations are easy. The old method required an IP Address for each and every site and logical site and a web server could only have one SSL cert. Not great and based on the old CERN server. We say legacy but most companies that do not use the great features of Domino will be on this type of config.

Domino can also use site documents and then it uses HTTP 1.1 headers and persistent connections or both. This is much closer to the Apache server and other modern web servers. With this configuration you can assign a separate SSL cert for each logical site plus a whole lot more such as url redirection etc.

To create a SSO config a web SSO document(s) must be created for either the legacy mode or 1.1 mode. In essence the difference between the two is simply entering the Organisation or not.

If there is no entry in Organisation field it becomes the legacy document listed in Web Configurations and first in the view called * - Web SSO Configurations.

If there is an organisation listed then it can be found Internet Sites and Web SSO configuration: Ltpa token name.

Please note that even some IBM products still require legacy mode for legitimate technical reasons so a hybrid model is most common meaning that you will have two SSO documents. This will really bite you if not careful. Also, forget about shortcuts. Once the configuration is complete you must reboot all servers to make SSO work properly.

Some may argue here and let me tell you from experience that when you go on site and somebody messed with the SSO document without your knowledge and a server is rebooted and the dreaded SSO Invalid message comes up you will get a call at stupidaclock saying you said ICM and failover will work and it does not and now a 1000 angry bees can’t access iNotes email. Buzzz, buzzzz, sting, sting, ouch!

This is taking longer than expected and I am ranting so here is how you should have isolated the issue.
  1. Start Domino and check for the entry Web SSO loaded after the HTTP server is started.
  2. Use Firefox and clear all cookies
  3. Sign on to a Domino web database and check the cookies.
  4. There should be one domain entry with two, if configured for Ltpatoken2, entries.
  5. Now open a database on the Sametime or Quickr server that requires authentication
  6. You should be automatically signed in
  7. Now access the Portal URL. You should be signed in automatically.
  8. Tell the Portal admin to fix the Portal config and go have a tea break
To redo the config simply import the WAS SSO Key. You do not have to recreate the document as the key will not expire. The key expiry is set to minutes, 300 recommended, and simply tells the cookie when it should expire and challenge you for credentials.

Simples hey!

Some more reading for you:

Thursday, 4 November 2010

IBM i and Enterprise upgrades - Pex Trace

Running a PEX Trace

If you are working on an IBM i (iSeries, AS/400) you can run a PEX trace. This will require a temporary license if you do not have it.

Performance Explorer (PEX) also known as iDoctor will collect all requests to CPU on a system.

This information can be reviewed to determine which processes consume the most CPU on the System.

To collect the data, a 5250 session to the system under review must be available.

1. First a collection must be defined.

Here are some examples:


The top command creates a new Performance collection called BESTEST, This command will run the trace against all jobs on the system. You can optionally choose to trace a single job by modifying the JOB parameter.

ADDPEXDFN DFN(BESTEST) TYPE(*TRACE) JOB((*ALL)) TASK(*ALL) MAXSTG(500000) INTERVAL(1) TRCTYPE(*SLTEVT) SLTEVT(*YES) BASEVT((*PMCO *NONE *FORMAT2))

ADDPEXDFN DFN(TPROF_CPU) TYPE(*TRACE) JOB(*ALL) TASK(*ALL) MAXSTG(100000) INTERVAL(1) TRCTYPE(*SLTEVT) SLTEVT(*YES) BASEVT(*PMCO) TEXT('TPROF PEX Trace')

ADDPEXDFN DFN(TPROFRC5I) TYPE(*TRACE) TASK(*ALL) MAXSTG(100000) INTERVAL(1) TRCTYPE(*SLTEVT) SLTEVT(*YES) BASEVT((*PMCO))

ADDPEXDFN DFN(MYCORPTASKSWT) TYPE(*TRACE) JOB((*ALL)) TASK(*ALL) MAXSTG(500000) TRCTYPE(*TASKSWT) TEXT('TASKSWT Pex Trace')

ADDPEXDFN DFN(MYCORPTASKSWT) TYPE(*TRACE) JOB((*ALL)) TASK(*ALL) MAXSTG(500000) TRCTYPE(*TASKSWT) TEXT('Task switch trace for pnovec contention')

ADDPEXDFN DFN(TPROF_CPU) TYPE(*TRACE) JOB((*ALL)) TASK(*ALL) MAXSTG(500000) INTERVAL(1) TRCTYPE(*SLTEVT)
SLTEVT(*YES) BASEVT((*PMCO)) TEXT('Tprof pex')

These definitions only need to be created once. Unless development changes the PEX criteria these do not need to be updated or recreated.

2. Create the libraries to store the data

CRTLIB nameoflib

3. Start the data collection.

When a slowdown occurs or when ready you need to start the PEX trace

STRPEX SSNID(trac1210) DFN(BESTEST)
STRPEX SSNID(nameoflib) DFN(TPROF_CPU)
STRPEX SSNID(nameoflib2) DFN(MYCORPTASKSWT)
STRPEX SSNID(SLOWDOWN) DFN(MYCORPTASKSWT)

This command starts the collection.

4. When the slowdown passes wait 5 to 10 minutes end the trace.

ENDPEX SSNID(trac1210)
ENDPEX SSNID(nameoflib) DTALIB(nameoflib)
ENDPEX SSNID(nameoflib2) DTALIB(nameoflib2)
ENDPEX SSNID(SLOWDOWN) DTALIB(SLOWDOWN) TEXT('SLOWDOWN ON LPAR ')

This command ends the trace, note this may take a long time to end. To avoid performance impact the trace is designed to have minimal impact during collection, as a result ending the trace is costly.

5. Once the PEX Trace has been ended the data needs to be sent to IBM

After ending the trace, you now have a collection in the library QPEXDATA to review. This library must be saved and sent to IBM for review. Here is an example. Your settings may differ.

Substitute the SLOWDOWN with the name of your Library.

wrklib
CRTSAVF FILE(QGPL/SLOWDOWN) TEXT('Save file for PEX trace')
SAVLIB LIB(SLOWDOWN) DEV(*SAVF) SAVF(QGPL/SLOWDOWN)
ftp testcase.software.ibm.com
anonymous
youremail@ibm.co.uk
bin
cd as400
cd toibm
lcd qgpl
put qgpl/slowdown slowdown.savf

Sending to support? Here is what IBM will do:

The data is sent to IBM rochester where it is loaded on any of four systems.

Install iDoctor for IBM i. You can find it here with its requirements:

https://www-912.ibm.com/i_dir/idoctor.nsf/downloadoptions.html

Start the iDoctor client and connect to the system you restored the data on with the correct authority.

Now analyze the PEX trace using the tools and interpret the data. Sounds easy?

In summary

CRTLIB TPROFSLOW
STRPEX SSNID(TPROFCPU) DFN(TPROF_CPU)
ENDPEX SSNID(TPROFCPU) DTALIB(TPROFSLOW) TEXT('TPROF DATA ON LPAR2p2 ')


How to take a Performance Explorer (PEX) trace on a single Domino job on IBM

1. Identify the Domino job utilizing a lot of CPU. Write down the job name, job user, and job number. For example:

job name = HTTP job user = QNOTES job number = 123456

2. Create a library to store the PEX data using the following command:

CRTLIB LIB(PEXHOLD)

3. Create a PEX definition specific to the job identified in step 1:

ADDPEXDFN DFN(JOBCPU) TYPE(*TRACE) JOB((123456/QNOTES/HTTP)) MAXSTG(500000) INTERVAL(1) TRCTYPE(*SLTEVT) SLTEVT(*YES) BASEVT((*PMCO *NONE *FORMAT2))

4. Start the PEX trace. Let the trace run for 3 to 5 minutes while the CPU is high.

STRPEX SSNID(CPU1) DFN(JOBCPU)

5. End the PEX trace. Because the ENDPEX can run a long time, it is recommended to run this command in batch:

SBMJOB CMD(ENDPEX SSNID(CPU1) DTALIB(PEXHOLD)) JOB(ENDPEX)

6. Wait for the ENDPEX batch job to end. Use the WRKSBMJOB to monitor the progress of the job

7. Create a save file to store the data, where xxxxx is your open Lotus PMR number:

CRTSAVF FILE(QGPL/PMRxxxxx)

8. Save the PEX library to the save file created in the previous step:

SAVLIB LIB(PEXHOLD) DEV(*SAVF) SAVF(QGPL/PMRxxxxx)

9. Send this save file to Lotus Support for review.

Capture the following as well:
  • wrksysval qmodel
  • wrksyval qprcfeat
  • print screen of wrksyssts
  • print screen of wrkdsksts