Session 1 and X - Data Management and Acquisition

Mail questions to: todd.valentic@sri.com

What are SOA's - service oriented architectures
    - computers talking to computers for accessing data from websites
    - i.e. catalog service
    - soa is an approach, web services are an implementation

Need to educate community about technologies and techniques
    - Get to know CADIS team
    - Work with PI's for web services - they have the knowledge for
        QC, but not expertise on technical side
    - Need feedback mechanisms - reprocessing, how to transmit
        issues and errors back to source
    - Other critera for data collected about humans
        sensitivity issues
        authorization
        hipa?
        identity exposure via metadata alone

Classes of data
    - 0 - raw, 1 - first pass, 2 - best quality (final)
    - "regridded" or output data for models, etc
    - data from outside
    - model output
        - how to treat?
        - AON-based model efforts
        - what type metadata (CF)
        - can they be treated just like data from the network?
        - exposing ancillary fields in model data
    - CADIS will support wide range of data types
        - more broad than just point observations
        - metadata closley works with other sources
    - Reanalysis of weather data
    - Authorization 
        - who?
        - public domain?
        - PI defined groups?
        - AON requirement?
        - authenticating vs autherization 
        - Need to track who is coming in and who is using data
        - What are the community policies?
            - can we track? 
            - do we need to track?
        - Feedback is useful
        - PI and users protected for problems in data
        - machine to machine access impacts?
    - Socialization aspects
        - "embargoed" data
        - policy, rules of the road
        - responsibility to contact data PIs (ie before publication)
            to make sure data are correct, latest, etc
        - example - IPCC climate change model runs
            - embargos, authentication
            - barrier, moving from social agreement to technical impl.
        - problem is people want to hold on to data until it is ok


Is AON SEARCH?
    - What is the take of the SEARCH data management group
        - talk this afternoon
    - They are intersecting circles
    - AON is a fundamental component of SEARCH
    - AON is concrete in producing data'
    - AON will influence SEARCH data management plan
    - What is the scope of the problem?
    - AON will collect data and use data from others
        - need to work with other data providers
        - need to know what you need and how to access it
        - do you want to send people to other places? risk of loosing them
    - Is AON providing a portal or VO?
    - Connecting to others means knowing about the issues necessary
        to integrate with them
        - GEOS
        - endoresments, but no funding
        - how much accomodation

Priorities and directions
    - Need something soon
    - How to handle data now
    - What data exists and where to get it 
        - good first step
        - then proceed to services
    - Centralized metadata? 
        registry - metadata, location of datasets, inventory
    - Where is the metadata portal accessible? 
        will others be able to find it?
    - Importance of taking care of data - NSF's investment
        - tension between caring and access? 
        - how to make data useful to broader community and stakeholders?

Metadata
    - level of service
        - more info provided, better/more things we can do

THREDDS
    - approach to inventory
    - appears on portals, services
    - human and machine access
    - harvestable by GCMD
        - automatically search against other data sets

Q7 - Data assimilation and fusion integration
    - Important for regulatity in space and time for effective modelling
    - Use of old climatologies
    - Where to get most up to date datasets
    - Push (RSS) vs pull (FTP) data sets
        - Event based observing

Data Policies -
    - more liberal policies better?
    - IPY data policy
        - liberal with restrictions for IP, privacy issues
        - define data (projects in honeycomb)
    - SEARCH policy
        - builds off IPY
    - Does AON follow SEARCH policy?
        - for the moment yes
        - AON is integral with SEARCH
        - can evolve as necessary
    - Should be made as quickly as available
    - Delay in data access reduces usefulness (ie forecasting)
    - Credit for collecting data
        - first pub
        - public access enough?
        - make access equal to publications
    - Need for attitude changes
    - Formal citation for data
    - Use IPY as an opportunity to try new things!

Assimilation models for sea ice thickness

Access and archive legacy information
    - North slope
    - Who to contact
    - Difficulty in finding NSF legacy data/PIs