ࡱ> #0( / 0LDArialЃ 0Ѓ0hhxz 0@ .  @n?" dd@  @@`` : 2 tY       0e0e     A@  A5% 8c8c     ?1 d0u0@Ty2 NP'p<'pA)BCD|E||s " 0e@        @ABC DEEFGHIJK5%LMNOPQRSTUWYZ[ \]^_ `abN E5%  N E5%  N F   5%    !"?N@ABC DEFFGHIJK5%LMNOPQRSTUWYZ[ \]^_ `ab@3ʚ;ʚ;g4`d`d! 0ppp@ <4dddd4w 0hTz 80___PPT10 " LEAD-VGrADS Day 1 Notes Introduction & discussiondWorkflow discussion Interaction points Higher level resource requirements Where does bind happen? WCS is being developed right now, timeline not clear Does vgLaunch need to be retrofitted for a web service call? LEAD is BPEL service based  for FT, adaptation, etc GT4 toolkit is also web service based 3P3lLEAD is moving to GT4 WS-GRAM, notifications are in different format from LEAD workflow LEAD has focus on deployment and integration, very application focused LEAD might use more than one scheduler/resource solutions ISI (Ann/Ewa) are working with LEAD on the data side (DRS, RLS), domain side (THREDS) WCS handles application configuration such as namelist files mZm,xWhat do each of us need??Interfaces between LEAD and VGrADS? Performance Model? What in LEAD communicates with VGrADS? And what does it give to VGrADS? Interfaces between vgES and batch queue prediction What requirements does LEAD have and does it conflict with vgES philosophy? New features to implement in vgES inline with deadline for SC06 @Z@t4W*!Short-term and long-term integration plan How do I use vgES? From LEAD workflow  concrete example of LEAD workflow? How to use batch queue from vgES? Batch queue waiting time aware Interface points are crucial Full ahead planning  resources might not be available at this time. In LEAD the first few tasks are light, last are intensive. So not much time lead time LEAD today will launch immediately. What if you start running before or after you expect to? P,7W<VThere is post-processing steps once we know what resources are needed Each workflow task  needs to be on a cluster, will need shared filesystem, good network connectivity Practical resource constraints>Where does and where can LEAD run today? (What are the real resource choices) WRF does not run on Itanium2 Have dependencies lot of libraries, binaries, compilers Today NCSA machines: copper, tungsten SDSC: tglogin IU teragrid PSC (luemix?) LEAD testbed (?) All GridFTP and GRAM enabled How about UC and UT machines? Possibilities in vgES UNC RTC supports GRAM but is Itanium2 Rice has another cluster. Can it be made Grid aware? Other sites? Credentials under which account  drlead account. vgES developers get access to drlead NP2P&PP[PTPkPYPPN2&   [  TkY  C We need to use LEAD workflow engine Scheduler makes assumption that queue prediction is accurate for first architecture interactions. What do we do when we are not accurate? vgES select and bind is tight Dan s perl workflow doesn t have notion of when to launch Having a deadline changes the way we look at this We need to submit job right at some time based on current wait queue times Prediction is used for which queue to submit, cannot use that to predict (selection happens apriori) Can t guarantee when we can tell where to run resources What is the interface Scheduler is static right now. Is dynamic scalable?  P2PKPP4PP 2K5  >   UNC/Lavanya will have to provide Suresh a standard interface to ask for resources and to submit jobs. Lavanya will turn around and do vgLaunch Data transfer is tightly coupled with computation. Can it be considered as a separate task? It is handled by myLEAD independently vgLaunch might or not do WS-GRAM How will we get the WS-GRAM notifications back through vgES? Application service is doing file staging (RFT didn t seem to work right now) Notifications for files that are generated is based on job completion (this will need to change with file streaming) Data mining, visualization, etc Does vgMon help with some notification? D6PIPP6IboQ  LEAD Data Management.WE first registers the topic ID with myLEAD LEAD publishes events on the data status (task A has input data) myLEAD agent contacts DRS (which is managing the data) and tells it what to do with the data When a task gets a unique id (resolves to GridFTP) it resolves to the best possible input source //>%B3    After LunchDemo Few slides from Suresh to understand data parts Service map of application service Workflow DAG More about the interfaces v /P          0` 33` Sf3f` 33g` f` www3PP` ZXdbmo` \ғ3y`Ӣ` 3f3ff` 3f3FKf` hk]wwwfܹ` ff>>\`Y{ff` R>&- {p_/̴>?" dd@,|?" dd@   " @ ` n?" dd@   @@``PR    @ ` ` p>> f(    6  `} z T Click to edit Master title style! !  0T  ` z RClick to edit Master text styles Second level Third level Fourth level Fifth level!     S  0 ^ ` z >*  0| ^  z @*  0| ^ ` z @*H  0޽h ? 3380___PPT10.b@{M Default Design 0 zr0  (     0` P    P*    0,y     R*  d  c $ ?    0h  0  RClick to edit Master text styles Second level Third level Fourth level Fifth level!     S  6L _P   P*    6h _   R*  H  0޽h ? 3380___PPT10.bp\$ 0  $(  r  S p!> z r  S ! `   z H  0޽h ? 3380___PPT10.bB}M$  0 ,$(  ,r , S t `}   r , S  `  H , 0޽h ? 3380___PPT10.b<.$  0 4$(  4r 4 S l `}   r 4 S \ `  H 4 0޽h ? 3380___PPT10.bS$  0 P$(  r  S Where does and where can LEAD run today? (What are the real resource choices) WRF does not run on Itanium2 Have dependencies lot of libraries, binaries, compilers Today NCSA machines: copper, tungsten SDSC: tglogin IU teragrid PSC (luemix?) LEAD testbed (?) All GridFTP and GRAM enabled How about UC and UT machines? Possibilities in vgES UNC RTC supports GRAM but is Itanium2 Rice has another cluster. Can it be made Grid aware? Other sites? Credentials under which account  drlead account. vgES developers get access to drlead NP2P&PP[PTPkPYPPN2&   [  TkY  C We need to use LEAD workflow engine Scheduler makes assumption that queue prediction is accurate for first architecture interactions. What do we do when we are not accurate? vgES select and bind is tight Dan s perl workflow doesn t have notion of when to launch Having a deadline changes the way we look at this We need to submit job right at some time based on current wait queue times Prediction is used for which queue to submit, cannot use that to predict (selection happens apriori) Can t guarantee when we can tell where to run resources What is the interface Scheduler is static right now. Is dynamic scalable?  P2PKPP4PP 2K5  >   UNC/Lavanya will have to provide Suresh a standard interface to ask for resources and to submit jobs. Lavanya will turn around and do vgLaunch Data transfer is tightly coupled with computation. Can it be considered as a separate task? It is handled by myLEAD independently vgLaunch might or not do WS-GRAM How will we get the WS-GRAM notifications back through vgES? Application service is doing file staging (RFT didn t seem to work right now) Notifications for files that are generated is based on job completion (this will need to change with file streaming) Data mining, visualization, etc Does vgMon help with some notification? D6PIPP6IboQ  LEAD Data Management.WE first registers the topic ID with myLEAD LEAD publishes events on the data status (task A has input data) myLEAD agent contacts DRS (which is managing the data) and tells it what to do with the data When a task gets a unique id (resolves to GridFTP) it resolves to the best possible input source //>%B3    After LunchDemo  !"#$%&'()*+,-./0123456789:;<=>ZABCDEFGHIJKLMNOPQRSTUVvY[\]^_`abcdefghijklmnopq@tu?wxz{|}~Root EntrydO)@bsCurrent User)SummaryInformation(`PowerPoint Document(3DDocumentSummaryInformation8alЃ 0Ѓ0hhxz 0@ .  @n?" dd@  @@`` : 2 tY       0e0e     A@  A5% 8c8c     ?1 d0u0@Ty2 NP'p<'pA)BCD|E||s " 0e@        @ABC DEEFGHIJK5%LMNOPQRSTUWYZ[ \]^_ `abN E5%  N E5%  N F   5%    !"?N@ABC DEFFGHIJK5%LMNOPQRSTUWYZ[ \]^_ `ab@3ʚ;ʚ;g4`d`d! 0ppp@ <4dddd4w 0hTz 80___PPT10 " LEAD-VGrADS Day 1 Notes Introduction & discussiondWorkflow discussion Interaction points Higher level resource requirements Where does bind happen? WCS is being developed right now, timeline not clear Does vgLaunch need to be retrofitted for a web service call? LEAD is BPEL service based  for FT, adaptation, etc GT4 toolkit is also web service based 3P3lLEAD is moving to GT4 WS-GRAM, notifications are in different format from LEAD workflow LEAD has focus on deployment and integration, very application focused LEAD might use more than one scheduler/resource solutions ISI (Ann/Ewa) are working with LEAD on the data side (DRS, RLS), domain side (THREDS) WCS handles application configuration such as namelist files mZm,xWhat do each of us need??Interfaces between LEAD and VGrADS? Performance Model? What in LEAD communicates with VGrADS? And what does it give to VGrADS? Interfaces between vgES and batch queue prediction What requirements does LEAD have and does it conflict with vgES philosophy? New features to implement in vgES inline with deadline for SC06 @Z@t4W*!Short-term and long-term integration plan How do I use vgES? From LEAD workflow  concrete example of LEAD workflow? How to use batch queue from vgES? Batch queue waiting time aware Interface points are crucial Full ahead planning  resources might not be available at this time. In LEAD the first few tasks are light, last are intensive. So not much time lead time LEAD today will launch immediately. What if you start running before or after you expect to? P,7W<VThere is post-processing steps once we know what resources are needed Each workflow task  needs to be on a cluster, will need shared filesystem, good network connectivity Practical resource constraints>Where does and where can LEAD run today? (What are the real resource choices) WRF does not run on Itanium2 Have dependencies lot of libraries, binaries, compilers Today NCSA machines: copper, tungsten SDSC: tglogin IU teragrid PSC (luemix?) LEAD testbed (?) All GridFTP and GRAM enabled How about UC and UT machines? Possibilities in vgES UNC RTC supports GRAM but is Itanium2 Rice has another cluster. Can it be made Grid aware? Other sites? Credentials under which account  drlead account. vgES developers get access to drlead NP2P&PP[PTPkPYPPN2&   [  TkY  C We need to use LEAD workflow engine Scheduler makes assumption that queue prediction is accurate for first architecture interactions. What do we do when we are not accurate? vgES select and bind is tight Dan s perl workflow doesn t have notion of when to launch Having a deadline changes the way we look at this We need to submit job right at some time based on current wait queue times Prediction is used for which queue to submit, cannot use that to predict (selection happens apriori) Can t guarantee when we can tell where to run resources What is the interface Scheduler is static right now. Is dynamic scalable?  P2PKPP4PP 2K5  >   UNC/Lavanya will have to provide Suresh a standard interface to ask for resources and to submit jobs. Lavanya will turn around and do vgLaunch Data transfer is tightly coupled with computation. Can it be considered as a separate task? It is handled by myLEAD independently vgLaunch might or not do WS-GRAM How will we get the WS-GRAM notifications back through vgES? Application service is doing file staging (RFT didn t seem to work right now) Notifications for files that are generated is based on job completion (this will need to change with file streaming) Data mining, visualization, etc Does vgMon help with some notification? D6PIPP6IboQ  LEAD Data Management.WE first registers the topic ID with myLEAD LEAD publishes events on the data status (task A has input data) myLEAD agent contacts DRS (which is managing the data) and tells it what to do with the data When a task gets a unique id (resolves to GridFTP) it resolves to the best possible input source //>%B3    After LunchDemo Few slides from Suresh to understand data parts Service map of application service Workflow DAG More about the interfaces /P       r} s}1|0(  Oh+'00 `h|   LEAD-VGrADSUNCUNC121Microsoft Office PowerPoint@Q@05Ub@{b G8g    y--$xx--'@Arial-.  2 6/LEAD."System-@Arial-.  2 6I-7.-@Arial-. 2 6LVGrADS.-@Arial-. 2 K= Day 1 Notes.-՜.+,0t    On-screen ShowUNC3Df  G ArialDefault Design LEAD-VGrADSIntroductio Few slides from Suresh to understand data parts Service map of application service Workflow DAG What do you want to see the demo part? More about the interfaces (/P       $   0 p$(  pr p S  `}   r p S H `  H p 0޽h ? 3380___PPT10.bPX;rԭX 1|0( / 0LDArialЃ 0Ѓ0hhxz 0@ .  @n?" dd@  @@`` : 2 tY       0e0e     A@  A5% 8c8c     ?1 d0u0@Ty2 NP'p<'pA)BCD|E||s " 0e@        @ABC DEEFGHIJK5%LMNOPQRSTUWYZ[ \]^_ `abN E5%  N E5%  N F   5%    !"?N@ABC DEFFGHIJK5%LMNOPQRSTUWYZ[ \]^_ `ab@3ʚ;ʚ;g4`d`d! 0ppp@ <4dddd4w 0hTz 80___PPT10 '# LEAD-VGrADS Day 1 Notes Introduction & discussiondWorkflow discussion Interaction points Higher level resource requirements Where does bind happen? WCS is being developed right now, timeline not clear Does vgLaunch need to be retrofitted for a web service call? LEAD is BPEL service based  for FT, adaptation, etc GT4 toolkit is also web service based 3P3lLEAD is moving to GT4 WS-GRAM, notifications are in different format from LEAD workflow LEAD has focus on deployment and integration, very application focused LEAD might use more than one scheduler/resource solutions ISI (Ann/Ewa) are working with LEAD on the data side (DRS, RLS), domain side (THREDS) WCS handles application configuration such as namelist files mZm,xWhat do each of us need??Interfaces between LEAD and VGrADS? Performance Model? What in LEAD communicates with VGrADS? And what does it give to VGrADS? Interfaces between vgES and batch queue prediction What requirements does LEAD have and does it conflict with vgES philosophy? New features to implement in vgES inline with deadline for SC06 @Z@t4W*!Short-term and long-term integration plan How do I use vgES? From LEAD workflow  concrete example of LEAD workflow? How to use batch queue from vgES? Batch queue waiting time aware Interface points are crucial Full ahead planning  resources might not be available at this time. In LEAD the first few tasks are light, last are intensive. So not much time lead time LEAD today will launch immediately. What if you start running before or after you expect to? P,7W<VThere is post-processing steps once we know what resources are needed Each workflow task  needs to be on a cluster, will need shared filesystem, good network connectivity Practical resource constraints>Where does and where can LEAD run today? (What are the real resource choices) WRF does not run on Itanium2 Have dependencies lot of libraries, binaries, compilers Today NCSA machines: copper, tungsten SDSC: tglogin IU teragrid PSC (luemix?) LEAD testbed (?) All GridFTP and GRAM enabled How about UC and UT machines? Possibilities in vgES UNC RTC supports GRAM but is Itanium2 Rice has another cluster. Can it be made Grid aware? Other sites? Credentials under which account  drlead account. vgES developers get access to drlead NP2P&PP[PTPkPYPPN2&   [  TkY  C We need to use LEAD workflow engine Scheduler makes assumption that queue prediction is accurate for first architecture interactions. What do we do when we are not accurate? vgES select and bind is tight Dan s perl workflow doesn t have notion of when to launch Having a deadline changes the way we look at this We need to submit job right at some time based on current wait queue times Prediction is used for which queue to submit, cannot use that to predict (selection happens apriori) Can t guarantee when we can tell where to run resources What is the interface Scheduler is static right now. Is dynamic scalable?  P2PKPP4PP 2K5  >   UNC/Lavanya will have to provide Suresh a standard interface to ask for resources and to submit jobs. Lavanya will turn around and do vgLaunch Data transfer is tightly coupled with computation. Can it be considered as a separate task? It is handled by myLEAD independently vgLaunch might or not do WS-GRAM How will we get the WS-GRAM notifications back through vgES? Application service is doing file staging (RFT didn t seem to work right now) Notifications for files that are generated is based on job completion (this will need to change with file streaming) Data mining, visualization, etc Does vgMon help with some notification? D6PIPP6IboQ  LEAD Data Management.WE first registers the topic ID with myLEAD LEAD publishes events on the data status (task A has input data) myLEAD agent contacts DRS (which is managing the data) and tells it what to do with the data When a task gets a unique id (resolves to GridFTP) it resolves to the best possible input source //>%B3    After LunchDemo Few slides from Suresh to understand data parts Service map of application service Workflow DAG What do you want to see the demo part? More about the interfaces (/P       r D1K0( / 0LDArialЃ 0Ѓ0hhxz 0@ .  @n?" dd@  @@`` : 2 tY       0e0e     A@  A5% 8c8c     ?1 d0u0@Ty2 NP'p<'pA)BCD|E||s " 0e@        @ABC DEEFGHIJK5%LMNOPQRSTUWYZ[ \]^_ `abN E5%  N E5%  N F   5%    !"?N@ABC DEFFGHIJK5%LMNOPQRSTUWYZ[ \]^_ `ab@3ʚ;ʚ;g4`d`d! 0ppp@ <4dddd4w 0hTz 80___PPT10 " LEAD-VGrADS Day 1 Notes Introduction & discussiondWorkflow discussion Interaction points Higher level resource requirements Where does bind happen? WCS is being developed right now, timeline not clear Does vgLaunch need to be retrofitted for a web service call? LEAD is BPEL service based  for FT, adaptation, etc GT4 toolkit is also web service based 3P3lLEAD is moving to GT4 WS-GRAM, notifications are in different format from LEAD workflow LEAD has focus on deployment and integration, very application focused LEAD might use more than one scheduler/resource solutions ISI (Ann/Ewa) are working with LEAD on the data side (DRS, RLS), domain side (THREDS) WCS handles application configuration such as namelist files mZm,xWhat do each of us need??Interfaces between LEAD and VGrADS? Performance Model? What in LEAD communicates with VGrADS? And what does it give to VGrADS? Interfaces between vgES and batch queue prediction What requirements does LEAD have and does it conflict with vgES philosophy? New features to implement in vgES inline with deadline for SC06 @Z@t4W*!Short-term and long-term integration plan How do I use vgES? From LEAD workflow  concrete example of LEAD workflow? How to use batch queue from vgES? Batch queue waiting time aware Interface points are crucial Full ahead planning  resources might not be available at this time. In LEAD the first few tasks are light, last are intensive. So not much time lead time LEAD today will launch immediately. What if you start running before or after you expect to? P,7W<VThere is post-processing steps once we know what resources are needed Each workflow task  needs to be on a cluster, will need shared filesystem, good network connectivity Practical resource constraints>Where does and where can LEAD run today? (What are the real resource choices) WRF does not run on Itanium2 Have dependencies lot of libraries, binaries, compilers Today NCSA machines: copper, tungsten SDSC: tglogin IU teragrid PSC (luemix?) LEAD testbed (?) All GridFTP and GRAM enabled How about UC and UT machines? Possibilities in vgES UNC RTC supports GRAM but is Itanium2 Rice has another cluster. Can it be made Grid aware? Other sites? Credentials under which account  drlead account. vgES developers get access to drlead NP2P&PP[PTPkPYPPN2&   [  TkY  C We need to use LEAD workflow engine Scheduler makes assumption that queue prediction is accurate for first architecture interactions. What do we do when we are not accurate? vgES select and bind is tight Dan s perl workflow doesn t have notion of when to launch Having a deadline changes the way we look at this We need to submit job right at some time based on current wait queue times Prediction is used for which queue to submit, cannot use that to predict (selection happens apriori) Can t guarantee when we can tell where to run resources What is the interface Scheduler is static right now. Is dynamic scalable?  P2PKPP4PP 2K5  >   UNC/Lavanya will have to provide Suresh a standard interface to ask for resources and to submit jobs. Lavanya will turn around and do vgLaunch Data transfer is tightly coupled with computation. Can it be considered as a separate task? It is handled by myLEAD independently vgLaunch might or not do WS-GRAM How will we get the WS-GRAM notifications back through vgES? Application service is doing file staging (RFT didn t seem to work right now) Notifications for files that are generated is based on job completion (this will need to change with file streaming) Data mining, visualization, etc Does vgMon help with some notification? D6PIPP6IboQ  LEAD Data Management.WE first registers the topic ID with myLEAD LEAD publishes events on the data status (task A has input data) myLEAD agent contacts DRS (which is managing the data) and tells it what to do with the data When a task gets a unique id (resolves to GridFTP) it resolves to the best possible input source //>%B3    After LunchDemo Few slides from Suresh to understand data parts Service map of application service Workflow DAG What do you want to see the demo part? More about the interfaces  /P       $   0 p$(  pr p S  `}   r p S H `  H p 0޽h ? 3380___PPT10.bPX;rxA TC1Root EntrydO)D>bsCurrent UserMSummaryInformation(`PowerPoint Document(3DY  !"#$%&'()*+,-./0123456789:;<=>ZABCDEFGHIJKLMNOPQRSTUVv[\]^_`abcdefghijklmnopq@tuwxz{|}~ n & discussionSlide 3What do each of us need?Slide 5Slide 6Practical resource constraintsSlide 8Slide 9LEAD Data Management Slide 11 After Lunch  Fonts UsedDesign Template Slide Titles 8g  '_DCharles KoelbelCharles Koelbel--'@Arial