Tony Doyle - University of Glasgow
3 February 2005
Science Committee Meeting
GridPPStatus Report
Tony Doyle
Tony Doyle - University of Glasgow
3 February 2005
Science Committee Meeting
Contents
•What was GridPP1?
•What is GridPP2?
•Challenges abound
•LCG
–Issues
•Deployment Status(9-28-30/1/05)
–UK Grid
•The UK mountain climb
•Summary
Tony Doyle - University of Glasgow
3 February 2005
Science Committee Meeting
What was GridPP1?
•A team that built a workingprototype grid of significant scale
> 2,000 (9,000) CPUs
> 1,000 (5,000) TB of available storage
> 1,000 (6,000) simultaneous jobs
•A complex project where 88% of themilestones were completed and allmetrics were within specification
A Success
“The achievement of somethingdesired, planned, or attempted”
Tony Doyle - University of Glasgow
3 February 2005
Science Committee Meeting
Executive Summary I
•“The GridPP1 Project is now complete: following 3 yearsof development, a prototype Grid has been established,meeting the requirements of the experiments and fullyintegrated with LCG, currently the World’s largest Grid.Starting from this strong foundation, a more complexproject, GridPP2, has now started, with an extendedteam in the UK working towards a production Griddeployed for the benefit of all experiments by September2007.”
•We achieved (almost exactly) what we stated wewould do in building a prototype…
Tony Doyle - University of Glasgow
3 February 2005
Science Committee Meeting
Executive Summary II
•“2004 was a pivotal year, marked by extraordinary and rapid change withrespect to Grid deployment, in terms of scale and throughput. The scale ofthe Grid in the UK is more than 2000 CPUs and 1PB of disk storage (from atotal of 9,000 CPUs and over 5PB internationally), providing a significantfraction of the total resources required by 2007. A peak load of almost 6,000simultaneous jobs in August, with individual Resource Brokers able to handleup to 1,000 simultaneous jobs, gives confidence that the system should beable to scale up to the required 100,000 CPUs by 2007. A careful choice ofsites leads to acceptable (>90%) throughput for the experiments, but theinherent complexity of the system is apparent and many operationalimprovements are required to establish and maintain a production Grid ofthe required scale. Numerous issues have been identified that are now beingaddressed as part of GridPP2 planning in order to establish the requiredresource for particle physics computing in the UK.”
•Most projects fail in going from prototype toproduction…
•There are many issues: methodical approachreqd.
Tony Doyle - University of Glasgow
3 February 2005
Science Committee Meeting
What is GridPP2?
Structuresagreedand inplace(exceptLCGphase-2)
•253 Milestones, 112 Monitoring Metrics at present.
•Must deliver a “Production Grid”: robust, reliable, resilient, secure, stable servicedelivered to end-user applications.
•The Collaboration aims to develop, deploy and operate a very large Production Gridin the UK for use by the worldwide particle physics community.
Tony Doyle - University of Glasgow
3 February 2005
Science Committee Meeting
Must
share data between thousands of scientists with multipleinterests
link major (Tier-0 [Tier-1]) and minor (Tier-1 [Tier-2])computer centres
ensure all data accessible anywhere, anytime
grow rapidly, yet remain reliable for more than a decade
cope with different management policies of differentcentres
ensure data security
be up and running routinely by 2007
What are theGrid challenges?
Tony Doyle - University of Glasgow
3 February 2005
Science Committee Meeting
What are theGrid challenges?
Data Management,Security andSharing
1. Software process
2. Software efficiency
3. Deployment planning
4. Link centres
5. Share data
6. Manage data
7. Install software
8. Analyse data
9. Accounting
10. Policies
Tony Doyle - University of Glasgow
3 February 2005
Science Committee Meeting
Where do we start?Issues
https://edms.cern.ch/file/495809/2.2/LCG2-Limitations_and_Requirements.pdf
First large-scaleGrid productionproblemsbeingaddressed…
at all levels
“LCG-2 MIDDLEWAREPROBLEMS AND
REQUIREMENTS FORLHC EXPERIMENTDATA CHALLENGES”
Overall efficiency ~60% -> ~90%
¼ -> ½ of the problems
¾ -> ½ of the problems
Tony Doyle - University of Glasgow
3 February 2005
Science Committee Meeting
GridPP Deployment Status(9-28-30/1/05)
Three Grids on Global scale in HEP (similar functionality)
sitesCPUs
•LCG (GridPP)104 (16) 10,000 (2242)
•Grid3 [USA]292800
•NorduGrid303200
GridPPdeployment ispart of LCG
(Currently thelargest Grid inthe world)
The future Gridin the UK isdependent uponLCG releases
Tony Doyle - University of Glasgow
3 February 2005
Science Committee Meeting
UK Grid
The whole is better than
the sum of the parts..
Tony Doyle - University of Glasgow
3 February 2005
Science Committee Meeting
ApplicationsThere is a (slightly wonky?) wheelUse it to get to where you need to be
ZEUS uses LCG
•needs the Grid to respond toincreasing demand for MCproduction
• up to 6 million Geant events perweek on Grid since August 2004
1.The system developed forthe large LHC experimentsworks (more) effectively forother (less resource-intensive) applications
2.Experiments need to worktogether with deploymentteam/sites
3.The de-facto deploymentstandard is LCG – it ~works.We can add components asrequired, to meet eachexperiment’s needs
Tony Doyle - University of Glasgow
3 February 2005
Science Committee Meeting
Disseminationmuch has happened..more people are reading about it..
GridPP2 gets its first term reportFri 28 Jan 2005
BaBar UK moves into the Grid eraTue 11 Jan 2005
LHCb-UK members get up to speed with the GridWed 5 Jan 2005GridPP in PittsburghThu 9 Dec 2004GridPP website busier than everMon 6 Dec 2004Optorsim 2.0 releasedWed 24 Nov 2004ZEUS produces 5 million Grid eventsMon 15 Nov 2004CERN 50th anniversary receptionTue 26 Oct 2004GridPP at CHEP'04Mon 18 Oct 2004LHCb data challenge first phase a success for LCG and UKMon 4 Oct 2004Networking in Nottingham - GLIF launch meetingMon 4 Oct 2004GridPP going for Gold - website award at AHMMon 6 Sep 2004GridPP at the All Hands MeetingWed 1 Sep 2004R-GMA included in latest LCG releaseWed 18 Aug 2004LCG2 administrators learn tips and tricks in OxfordTue 27 Jul 2004Take me to your (project) leaderFri 2 Jul 2004ScotGrid's 2nd birthday: ScotGrid clocks up 1 million CPU hoursFri 25 Jun 2004Meet your production managerFri 18 Jun 2004GridPP10 report and photographsWed 9 Jun 2004CERN recognizes UK's outstanding contribution to Grid computingWed 2 Jun 2004UK particle physics Grid takes shapeWed 19 May 2004A new monitoring map for GridPPMon 10 May 2004Press reaction to EGEE launchTue 4 May 2004GridPP at the EGEE launch conferenceTue 27 Apr 2004LCG2 releasedThu 8 Apr 2004University of Warwick joins GridPPThu 8 Apr 2004
Grid computing steps up a gear: the start of EGEEThu 1 Apr 2004EDG gets glowing final reviewMon 22 Mar 2004Grids and Web Services meeting, 23 April, LondonTue 16 Mar 2004EU DataGrid Software License approved by OSIFri 27 Feb 2004GridPP Middleware workshop, March 4-5 2004, UCLFri 20 Feb 2004Version 1.0 of the Optorsim grid simulation tool released by EU DataGridTue 17 Feb 2004Summary and photographs of the 9th GridPP Collaboration MeetingThu 12 Feb 2004
138,976 hits
in December
Tony Doyle - University of Glasgow
3 February 2005
Science Committee Meeting
Annual data storage:
2.4-2.8 PetaBytes
per year? (~20%)
10 Million SPECint2000
10,000 PCs (3 GHz Pentium 4)
CD stack
(~ 4 km)
The UK mountain climbhas started..
Quantitatively, we’re ~10% of the way there in terms of
UK CPU (~2,000 ex ~10,000) and disk (~1 ex ~10 PB)
In production terms,left base camp
We are here
(0.4 km)
step-by-stepplan in place…
For the BenNevis climb?
Tony Doyle - University of Glasgow
3 February 2005
Science Committee Meeting
Summary GRIDPP-PMB-40-EXEC
•The Grid is a reality
•A project was/is needed
•Under control
•LCG2 support: SC case presn. 3/2/05
•16 UK sites are on the Grid
–MoUs, planning, deployment, monitoring
–each underway as part of GridPP2
•Developments estd., R-GMA deployed
•gLite designed inc. web services
•Interfaces developed, testing phase
•Area transformed
•Incorporation in HEP programme..
•Introduction
•Project Management
•Resources
•LCG
•Deployment
–Tier-1/A production
+Tier-2 resources
•M/S/N
•EGEE
•Applications
•Dissemination
•Beyond GridPP2
