Background:

More than 500 users (developers/testers) registered in RTC, and about 200 concurrent user access (looking at license consumption). Almost every evening, RTC server has slow response. Sometimes it look like stopping (not really but users think so). We've increased server resources (Linux, DB2, CPU=16/Memory=48GB/Network Bandwidth 75Mbps) and still haven't got enough. RTC workitem / scm / build features are used as well.

Examples:
1.Last year, we've got a hint from RCS that if "Pending Change sets" increases, it causes high server CPU load because of "compare workspace" feature done by RTC server - if 100 pending change sets, and there are 200 developer's repository workspaces, RTC will work 100x200 calculation every 15min(?) -> I made a rule to make BL frequently and announced to accept to users.
2.We made an operation guide for users, not to Load all component resources. One person of the group load all resources then burn a DVD and share with others, then others re-sync their repository workspace with RTC server. - Especially network environment in China is bad.
3.We use build feature to automate EAR creation, JUNIT/JTEST, those are done at another server, for not every users to do it on their own PC.

Question:

Rather than better sizing information / server performance tuning / Configuration (such as Load balance), I'm more interested in "wise use". What real RTC operations by number of users cause server resource load? What should be avoided? What is the way to find out the cause of slow server response? What user operations should I look at? Is it because of a bad operation rule? Is it the limit of server/network resource? What can I report to my boss as a reasonable cause? In order to avoid slow server performance in big projects, does anyone have any ideas for rules that I can make? All input will be very welcomed.

I would like to suggest:-

1. Check the number of repository (check my repository workspace) created by users. I have seen, most of the users creates many repository which is not used but as you know it stores on server.

2. think about keeping the large source files in Artifactory (only > CLM 6.0.5)

3. do you have any integration with other tools?? RTC=> ??)

Thanks & Best Regards,
/Manoj Panda
SAFe Program Consultant (SPC4)    
Certified Scrum Master, ITIL V3     
IBM Certified Deployment Professional     
Phone: +91 9618772582     
www.linkedin.com/in/manojkumar72
DevOps: Applying lean and agile principles across the software lifecycle

1.repository which is not used but as you know it stores on server.

I found a lot of personal repository workspaces. I'll delete them now and will delete every time users are archived as the new rule. Good idea.

2. think about keeping the large source files in Artifactory (only > CLM 6.0.5)

Sorry I couldn't catch well. We're using CLM 601so the version doesn't match, but would you let me know, for my study, if you don't mind.

3. do you have any integration with other tools?? RTC=> ??)

No tool integrations, quite normal usage. Only LDAP.

As per new Released Version of RTC 6.0.5, you can as well store the large size files in outside of RTC. for more info you may visit jazz.net.

You also need to check the DCC schedule as suggested.

Thanks & Best Regards,
/Manoj Panda
SAFe Program Consultant (SPC4)    
Certified Scrum Master, ITIL V3     
IBM Certified Deployment Professional     
Phone: +91 9618772582     
www.linkedin.com/in/manojkumar72
DevOps: Applying lean and agile principles across the software lifecycle

Masatoshi, you said that the problem occurs "Almost every evening"....don't you think that maybe RTC features might be concurring wiht jobs of DCC?

I checked DCC schedule. I think it won't be a cause of evening server side busyness.

  ODS data:0:00 once a day,

  Data mart: 3:00 once a day,

  License data: every 60 min