A.J. Chakravarti, Gerald Baumgartner, Mario Lauria
The Organic Grid is a biologically inspired and fully decentralized approach to the organization of computation that is based on the autonomous scheduling of strongly mobile agents on a peer-to-peer network. Through the careful design of agent behavior, the emerging organization of the computation can be customized for different classes of applications. In this paper, we report on our experience in adapting the general framework to run two representative applications on our Organic Grid prototype: the National Center for Biotechnology Information (NCBI) basic local alignment search tool (BLAST) code for sequence alignment, and the Cannon's algorithm for matrix multiplication. The first is an example of independent task application, a type of application commonly used for grid scheduling research because of its easily decomposable nature and absence of intra-node communication. The second is a popular block algorithm for parallel matrix multiplication, and represents a challenging application for grid platforms because of its highly structured and synchronous communication pattern. Agent behavior completely determines the way computation is organized on the Organic Grid. We intentionally chose two applications at opposite ends of the distributed computing spectrum having very different requirements in terms of communication topology, resource use, and response to faults. We detail the design of the agent behavior and show how the different requirements can be satisfied. By encapsulating application code and scheduling functionality into mobile agents, we decouple both computation and scheduling from the underlying grid infrastructure. In the resulting system, every node can inject a computation onto the grid; the computation naturally organizes itself around available resources.
Grid computing is an open, heterogeneous and highly dynamic environment based on the principles of service oriented computing. It focuses on basic infrastructure for coordinated resource sharing among virtual organizations to achieve high performance and availability. However, use of existing Grid computing environment is quite complex and requires a lot of human intervention. In order to avoid this intervention, enhancements are required in bringing autonomy and semantics in existing Grid infrastructure. Semantics would act as glue for autonomy in the process of efficient resource discovery and utilization. Several ontologies and ontology languages have been proposed in this regard which not only have some shortcoming but also poses a sort of overhead for the Grid environment. On the other hand, agents are autonomous problem solving entities, and can negotiate semantically for interoperation with each other in dynamic environments. Inspired from the concept of Autonomous Decentralized Systems, we propose that the above mentioned goals can be achieved by integrating FIPA Multi Agent Systems with the Grid Service Architecture and hence to lay the foundation for Autonomous Semantic Grid. Autonomous Semantic Grid system architecture is aimed to provide an improved infrastructure by bringing autonomy, semantic interoperability and decentralization in the Grid computing for emerging applications. This paper then presents implementation details of first milestone toward Autonomous Semantic Grid realization based on a middleware, namely AgentWeb Gateway for integration of Multi Agent Systems and Grid Service Architecture. Evaluation of the system has also been performed over a number of application scenarios.
Asynchronous collaborative applications and systems have to deal with complexities associated with interaction nature, idiosyncrasy of groups and technical and administrative issues. Inclusion of requirements derived from them is costly (in time, resources and economically). Existing solutions addresses asynchronous collaboration via simplification of requirements and by using centralized models. In this paper we present LaCOLLA, a fully decentralized infrastructure for building collaborative applications that provides general purpose collaborative functionalities. The provision of those functionalities will avoid applications deal with most of complexities derived from groups and its members, what will help inclusion of collaborative aspects. The implementation of LaCOLLA follows the peer-to-peer paradigm and pays special attention to autonomy of its members and to self-organization of the components of the infrastructure. Another key aspect is that resources (e.g. storage) and services (e.g. authorization) are provided by its members (avoiding dependency from agents not belonging to group).
A.J. Chakravarti, Gerald Baumgartner, Mario Lauria
We propose a biologically inspired and fully-decentralized approach to the organization of computation that is based on the autonomous scheduling of strongly mobile agents on a peer-to-peer network. Our approach achieves the following design objectives: near-zero knowledge of network topology, zero knowledge of system status, autonomous scheduling, distributed computation, lack of specialized nodes. Every node is equally responsible for scheduling and computation, both of which are performed with practically no information about the system. We believe that this model is ideally suited for large-scale unstructured grids such as desktop grids. This model avoids the extensive system knowledge requirements of traditional grid scheduling approaches. Contrary to the popular master/worker organization of current desktop grids, our approach does not rely on specialized super-servers or on application-specific clients. By encapsulating computation and scheduling behavior into mobile agents, we decouple both application code and scheduling functionality from the underlying infrastructure. The resulting system is one where every node can start a large grid job, and where the computation naturally organizes itself around available resources. Through the careful design of agent behavior, the resulting global organization of the computation can be customized for different classes of applications. In a previous paper, we described a proof-of-concept prototype for an independent task application. In this paper, we generalize the scheduling framework and demonstrate that our approach is applicable to a computation with a highly synchronous communication pattern, namely Cannon's matrix multiplication.
M. Jamshed, S. Khalique, Hiroki Suguri, Hafiz Farooq Ahmad · 5 authors
Grid is an ambitious and exciting global effort to develop an environment in which individual users can access computers, databases and experimental facilities simply and transparently, without having to consider where those faculties are located. Monitoring the behavior of the resources of a distributed system, especially grid, is necessary, both for determining the cause of performance problems, and for tuning the system, which results in optimized performance. However writing monitoring and management applications for grid resource is a complex task. This is due to the geographic distribution of resources and heterogeneous environments that are owned by different organizations. We have proposed architecture for autonomous grid node monitoring agent, to address the issues of complex resource monitoring and management. The architecture is based on grid monitoring architecture (GMA) and is autonomous as it provides functionality of equality, locality and self-containment, which are main components of autonomic controllability in any sub system as part of autonomous decentralized system (ADS) (K. Mori, 1993). The evaluation results of the proposed system proved the validity of the concept for achieving autonomous resource management in dynamic systems like grid
A.J. Chakravarti, Gerald Baumgartner, Mario Lauria
Desktop grids have recently been used to perform some of the largest computations in the world and have the potential to grow by several more orders of magnitude. However, current approaches to utilizing desktop resources require either centralized servers or extensive knowledge of the underlying system, limiting their scalability. We propose a biologically inspired and fully-decentralized approach to the organization of computation that is based on the autonomous scheduling of strongly mobile agents on a peer-to-peer network. In a radical departure from current models, we envision large-scale desktop grids in which agents autonomously organize themselves so as to maximize resource utilization. By encapsulating computation and behavior into agents, the organization of the computation can be customized for different classes of applications. At the same time, the design of the underlying infrastructure is greatly simplified, resulting in a system that naturally lends itself to a true peer-to-peer implementation where each node can be at the same time provider and user of the computing utility infrastructure. We demonstrate this concept with a reduced-scale proof-of-concept implementation that executes a data-intensive independent-task application on a set of heterogeneous, geographically distributed machines. We present a detailed exploration of the design space of our system and a performance evaluation of our implementation using metrics appropriate for assessing self-organizing desktop grids.
Current approaches to utilizing computational resources on large, unstructured networks like the Internet, require either centralized servers or extensive system knowledge, limiting their scalability. In a radical departure from current models, we follow a fully-decentralized approach in which mobile agents autonomously organize themselves on a peer-to-peer network so as to maximize resource utilization. Computation and scheduling behavior are encapsulated into agents and thus decoupled from the underlying infrastructure.
The following design objectives are achieved: near-zero knowledge of network topology, zero knowledge of system status, distributed computation, lack of specialized roles. Every node is equally responsible for scheduling and computation, both of which are performed with minimal information about the system. This model avoids the extensive system knowledge requirements of traditional Grid scheduling approaches. Unlike popular master/worker organizations of current large-scale grids, our model is not reliant on specialized super-servers or on application-specific clients.
Strong mobility allows the asynchronous and transparent migration of multi-threaded agents, and is a powerful mechanism for realizing a peer-to-peer computing environment. We have implemented and evaluated an agent system that endows Java with strong mobility.
Through the careful design of agent behavior, the global organization of computation can be customized for different classes of applications. We describe our scheduling framework and demonstrate the flexibility of our approach by applying it to two problems that are vastly different in terms of the synchrony of their communication patterns: an Independent Task Application and Cannon's Matrix Multiplication.
Arshad Ali, A. Anjum, Tahir Azim, M. Thomas · 9 authors
In this paper we describe JClarens; a Java based implementation of the Clarens remote data server. JClarens provides Web services for an interactive analysis environment to dynamically access and analyze the tremendous amount of data scattered across various locations. Additionally this research is aimed to develop a service oriented grid enabled portal (GEP) that provides interface and access to several grid services to give a homogeneous and optimized view of the distributed and heterogeneous environment. Other than showing platform independent behavior provided by Java, the use of XML-RPC based Web services enabled JClarens to be a language neutral server and demonstrated interoperability with its Python variant. Extreme care has been taken in the usage and manipulation of various Java libraries to cater the needs of high performance computing. The overall exercise has yielded in a prototype with strong emphasis on security and virtual organization management (VOM). This shall provide a common platform to support development of larger, more flexible framework with future aims to integrate it with a loosely coupled, decentralized, and autonomous framework for grid enabled analysis environment (GAE).
Opportunistic sharing of Internet-connected resources is a low cost method for obtaining access to unprecedented-scale collections of resources. An essential service in any resource-sharing environment is resource discovery: given a description of the resources desired, a resource discovery mechanism returns locations of resources that match the description.
Two resource-sharing environments are particularly well defined by applications, user communities, and deployments: Grid and peer-to-peer systems. Grids are sharing environments that rely on persistent, standards-based service infrastructures that allow well-established, mainly professional communities to share computers, storage space, sensors, software applications, and data across organizational boundaries. Peer-to-peer systems are Internet applications that harness resources from millions of autonomous participants. Thus, Grids provide infrastructure to support a variety of applications on resources shared by relatively small communities; at the scale of the peer-to-peer communities, remarkable sharing patterns are exhibited, such as free riding and intermittent resource participation.
The focus of this dissertation is on solution design for resource discovery in Grids of the scale and lack of reliability of today's peer-to-peer networks. This hybrid target environment requires fully decentralized solutions that scale with the number of users and resources and tolerate intermittent resource participation.
To explore the solution space, we propose a taxonomy for resource discovery solutions. This taxonomy proves to be a useful tool for discussing and comparing existing solutions.
Using this taxonomy, we delimit and explore a portion of the solution space. We build a scalable Grid emulator to evaluate mechanism performance in this subspace. Large-scale experiments reveal that the performance of mechanisms in this subspace is strongly dependent on sharing characteristics.
For inspiration, we turned to studying user behavior in various communities. We uncovered a significant usage pattern in file-sharing communities: users naturally form interest-based groups. This pattern can be exploited for system design in a variety of problems: we designed a file-location mechanism, FLASK, that exploits and benefits from this naturally emerging pattern. Trace-driven evaluations show FLASK leads to lower response latency, good scalability, support for intermittent participation; and satisfies requirements typical of scientific usage of data.
Grids enable users to share and access large collections and various types of resources in wide areas, and how to locate resources in such dynamic, heterogeneous and autonomous distributed environments is a key and challenging issue. In this paper, a three-level decentralized and dynamic VEGA Infrastructure for Resource Discovery (VIRD) is proposed. In this architecture, every Border Grid Resource Name Server (BGRNS) or Grid Resource Name Server (GRNS)has its own local policies, governing information organization, management and searching. Changes in resource information are propagated dynamically among GRNS servers according to a link-statelike algorithm. A client can query its designated GRNS either recursively or iteratively. Optimizing techniques, such as shortcut, are adopted to make the dynamic framework more flexible and efficient. A simulator called SimVIRD is developed to verify the proposed architecture and algorithms.Experiment results indicate that this architecture could deliver good scalability and performance for grid resource discovery.
FlexiInternational develops and maintains FlexiFinancials, a suite of object-oriented, client/server software products for accounting. These interacting applications support general ledger, accounts payable and receivable, purchasing, fixed assets, order processing, inventory maintenance, and workflow functions. This paper describes a distributed component, transaction-based architecture that comprises the foundation for future development.
This memo describes the Open Digital Asset Protocol (ODAP). ODAP is an
asset transfer protocol that operates between two gateway devices. The
protocol includes a description of virtual or digital assets held on
distributed ledgers in an open and interoperable format, a session
negotiation part and message passing flows between gateways connecting
disparate distributed ledger technologies (DLTs).
The focus of this research was the individual or small organization. These organizations include small businesses, community groups, K-12 schools or community colleges, local government, and the individual user, as well as many others. In this work, all of these organizations as well as the individual user were collectively referred to as users. The common element shared by each of these users was that they each have legitimate purposes for access to Internet services or each provides a service or services that could be enhanced if distributed via the connectivity provided by the Internet. However, the costs of establishing a conventional Internet server and the associated connectivity are prohibitive to such small-scale organizations. The objectives of this research were to: Establish a definition of a low-cost decentralized distributed computing environment for Intel-based personal computers that will provide users the capability to access the full spectrum of Internet services while enabling them with the ability to retain control of their computing environment. Develop a replication process to replicate and distribute the defined environment in a modular form so as to facilitate installation on a target system. Conduct testing and evaluation of the architecture and replication process to validate its ease of configuration and installation, and compliance with the requirements to provide users the capability to access the full spectrum of Internet services while retaining complete control of their computing environment. This was accomplished in three phases: (a) Phase I - Define an objective architecture, (b) Phase II - Develop a technique for replicating and distributing the architecture, and (c) Phase III - Test and validate the architecture and the replication and distribution processes. Definition of the objective architecture was accomplished through development of a prototype system that successfully demonstrated all of the characteristics required by the objectives of this research. Following the definition of the architecture on the prototype system, development of a technique for replicating and distributing the architecture was undertaken. This was accomplished by developing a group of programs that configured a system to the needs of a target user, captured that configured system on a removable medium, and restored that configured system on the target hardware. Finally the architecture, as well as its replication and distribution processes were evaluated for validity using statistical analysis of data collected from test subjects acting as users. All of these tasks were accomplished within the Linux Operating System environment using only software tools developed by the researcher or tools that are a native component of Linux. The first objective of this research was satisfied by the researcher's selection of Linux and its suite of associated applications as the operating system that would host the solution system. The second objective of this research was accomplished by the researcher's development of a suite of software tools that replicated the configured environment, moved the replication to an appropriate media and restored the environment on a target system. Inviting a group of Linux users to use the tools and provide feedback via a survey satisfied the third objective of this research. It was concluded that the three objectives of this research and therefore the overall goal of this research were accomplished. In each measured evaluation of the architecture, procedures and programs developed by the researcher, the resulting data were plotted in the advanced area or the area tending toward the advanced level of maturity as defined by the Boloix and Robillard (1995) evaluation scale. In a like manner the resulting data were plotted in the exceptionally compliant range or higher on the normal distribution curve survey scale. The trend of results was consistently at the advanced level of maturity on the Boloix and Robillard (1995) evaluation scale or in the exceptionally compliant range of the normal distribution curve survey scale. The researcher found that the results of testing the defined architecture and replication process revealed users are able to quickly implement a fully configured Linux system with all the capabilities defined in the architecture. This resulting Linux system provided a low cost, decentralized, distributed computing environment for Intel-based personal computers that enabled users to access the full spectrum of Internet services while maintaining control of their computing environment. By accomplishing this objective the researcher's Linux system can provide fiscally constrained individuals or small organizations full access to Internet services without the high costs of establishing a conventional Internet server and associated connectivity, prohibitive to a small-scale organization.
The autonomous decentralized systems (ADS) concept has been expanded to handle subsystem configuration on distributed computing environments effectively. The concept introduces the "group" as a logical set of subsystems based on application software or an organization. Subsystems on a "group" have an equal influence on the entire "group" and are hidden from the other subsystems. The "group" provides a simplified view of the system resource by hiding unrelated information from the users. Application management, monitoring and controlling services are provided on the basis of the "group". The expansion concept and a prototype tool ACP/EM are presented.>
The meteoric decline in hardware costs now in progress will result in computing becoming a personnel-intensive enterprise. Most of this personnel will be devoted to software development and support of computing users. The financing and budgeting in most university computing centers is centered on hardware and computing cycles. How, then, can university computing best adapt to the changing economic structure of the information processing industry?Who should provide user support services? The central computing service center? Individual departments and/or colleges? Are there significant economies of scale in user support? If there are, do they outweigh the responsiveness that can be provided by a decentralized network of user support services?Who should pay for user support services? The users? The university? If the university pays, should centralized or decentralized arrangements be made for the services?Who should determine the level of expenditure on user support services? The market? The departments and colleges? The central administration? The computing center management?What will be the policy implications of the alternate answers to the questions above? Will they enhance or retard the decentralization of computing services? Will they encourage users to depend on internal or external facilities? Will they lead to a growth or decline of user support activity within the computing center?All of the above questions are now and have been debated strenuously regarding the provision of computing cycles. There is every reason to believe that the same debate will take place regarding support services. Strong user support services will be the sine qua non of high quality university computing in the future, and it will not be available unless we deal promptly with the issues of pricing, budgeting, and organization outlined above.
The extraordinarily rapid advance of microelectronics is causing a rapid drop in both the cost and economy of scale of computing equipment. In less than a decade, the powerful economic incentive for centralizing the production of computing cycles has reversed. Simultaneously, the productivity of computing professionals has not kept pace with inflation, and it appears that staff productivity gains will be difficult to achieve in a decentralized environment.Distribution of computing equipment is making support for the individual computer user more difficult in the following ways:-- There is a need for support of multiple types of small computer hardware.-- There is a need to maintain multiple operating systems and versions of applications software.-- There is little knowledge and experience in providing documentation, user training, and consulting support in a highly distributed multi-architecture computing environment.This paper outlines Cornell's response to the challenge of decentralized computing. The paper details the rationale, planning, organization, financing, and staffing for the new Decentralized Academic Computer Support group in Cornell Computing Services. In addition, the responsibilities of that group and its interaction with the other groups in Computing Services is described. Finally, attention is given to the relationship between Computing Services and the Cornell community regarding small computers.The accelerating trend of decentralizing the production of computing cycles presents user support groups with both a challenge and an opportunity. Failure to respond will result in more expensive and lower quality computing for the university community as well as declining utilization of user support services.