• Title/Summary/Keyword: real-time job

Search Result 196, Processing Time 0.028 seconds

Job-aware Network Scheduling for Hadoop Cluster

  • Liu, Wen;Wang, Zhigang;Shen, Yanming
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.11 no.1
    • /
    • pp.237-252
    • /
    • 2017
  • In recent years, data centers have become the core infrastructure to deal with big data processing. For these big data applications, network transmission has become one of the most important factors affecting the performance. In order to improve network utilization and reduce job completion time, in this paper, by real-time monitoring from the application layer, we propose job-aware priority scheduling. Our approach takes the correlations of flows in the same job into account, and flows in the same job are assigned the same priority. Therefore, we expect that flows in the same job finish their transmissions at about the same time, avoiding lagging flows. To achieve load balancing, two approaches (Flow-based and Spray) using ECMP (Equal-Cost multi-path routing) are presented. We implemented our scheme using NS-2 simulator. In our evaluations, we emulate real network environment by setting background traffic, scheduling delay and link failures. The experimental results show that our approach can enhance the Hadoop job execution efficiency of the shuffle stage, significantly reduce the network transmission time of the highest priority job.

An Adaptively Speculative Execution Strategy Based on Real-Time Resource Awareness in a Multi-Job Heterogeneous Environment

  • Liu, Qi;Cai, Weidong;Liu, Qiang;Shen, Jian;Fu, Zhangjie;Liu, Xiaodong;Linge, Nigel
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.11 no.2
    • /
    • pp.670-686
    • /
    • 2017
  • MapReduce (MRV1), a popular programming model, proposed by Google, has been well used to process large datasets in Hadoop, an open source cloud platform. Its new version MapReduce 2.0 (MRV2) developed along with the emerging of Yarn has achieved obvious improvement over MRV1. However, MRV2 suffers from long finishing time on certain types of jobs. Speculative Execution (SE) has been presented as an approach to the problem above by backing up those delayed jobs from low-performance machines to higher ones. In this paper, an adaptive SE strategy (ASE) is presented in Hadoop-2.6.0. Experiment results have depicted that the ASE duplicates tasks according to real-time resources usage among work nodes in a cloud. In addition, the performance of MRV2 is largely improved using the ASE strategy on job execution time and resource consumption, whether in a multi-job environment.

An approximation method for job shop scheduling problem with sequence independent setup time (준비시간을 고려한 job shop 스케쥴링 문제의 근사적 해법에 관한 연구)

  • 정한일;김기동;정대영;박진우
    • Proceedings of the Korean Operations and Management Science Society Conference
    • /
    • 1996.04a
    • /
    • pp.306-309
    • /
    • 1996
  • The job shop scheduling problem has been a major target for many researchers. And, most of the past studies did not consider setup time. In many cases of real manufacturing environment, however, there exists a setup time for each operations. The setup can be divide into two parts, one can be done after job arrival. The setup time based on the latter can be summed together with processing time, but that based on the former can not be. We propose an approximation method based on shifting bottleneck procedure for solving the job shop scheduling problem with sequence independent setup time. It schedules the machines one by one, taking a bottleneck machine among the machines not yet scheduled. Every time after a new machine is scheduled, all schedules previously established are updated. Both the bottlenck search and the schedule updating procedure are based on solving a single machine scheduling problem with ready time, setup time and delivery time iteratively.

  • PDF

First Job Waiting Times after College Graduation Based on the Graduates Occupational Mobility Survey in Korea

  • Lee, Sungim;Moon, Jeounghoon
    • The Korean Journal of Applied Statistics
    • /
    • v.25 no.6
    • /
    • pp.959-975
    • /
    • 2012
  • Each year research institutions such as the Korea Employment Information Service(KEIS), a government institution established for the advancement of employment support services, and Job Korea, a popular Korean job website, announce first job waiting times after college graduation. This provides useful information understand and resolve youth unemployment problems. However, previous reports deal with the time as a completely observed one and are not appropriate. This paper proposes a new study on first job waiting times after college graduation set to 4 months prior to graduation. In Korea, most college students hunt for jobs before college graduation in addition, the full-fledged job markets also open before graduation. In this case the exact waiting time of college graduates can be right-censored. We apply a Cox proportional hazards model to evaluate the associations between first job waiting times and risk factors. A real example is based on the 2008 Graduates Occupational Mobility Survey(GOMS).

A Study on Smart Job Matching Platform for Personal Assistant of the Disabled (장애인 활동보조인의 구인·구직을 위한 비대면 플랫폼에 관한 연구)

  • Kang, Seungae
    • Convergence Security Journal
    • /
    • v.21 no.3
    • /
    • pp.75-80
    • /
    • 2021
  • This study provided personnel and recruitment information related to job offer and job hunting, and proposed a non-face-to-face platform that can facilitate matching of one-time services as well as periodic services. The existing recruitment and job search method for personal assistant of the disabled is a one-way matching method in which a person in charge of the service providing centers assigns an activity assistant belonging to the institution when a consumer requests an personal assistant service, so the choice of job seekers is very weak. The job matching platform of the activity assistant for the disabled can be logged in separately by dividing users and separating them into the interface of the disabled or the disabled family members (job seekers) and activity assistants for the disabled (job seekers). Search for job offer and job seekers that meet their respective conditions, and real-time bidirectional decision-making through the chat function, and when the activity assistance service is completed, recruiters and job seekers can write service reviews and evaluations from their own point of view. The real-time interactive decision-making matching platform will also help minimize blind spots where it is difficult to provide services when short-term or one-time assistance is needed.

Development of a production scheduling system for the real time controlled manufacturing system (실시간 제어가 가능한 일정 계획 시스템 개발)

  • 이철수;배상윤;이강주
    • Korean Management Science Review
    • /
    • v.10 no.2
    • /
    • pp.61-77
    • /
    • 1993
  • This paper involves a study of developing the production scheduling system in a general job shop type mechanical factory. We consider realistic situations in the job-shop environments, such as alternative machines for operation, the new kinds of processes, the machining center with the plural pallet, the operational situation of each machine during the scheduling period, occurings of urgent orders and machine breakdowns. We also propose the methodology of re-schedule. It is very fast and acceptable for real time production control system. These all functions are implemented on IBM PC and program source is written in PASCAL language.

  • PDF

Real-time Job Reallocation Problem in Container Terminals (컨테이너 터미널의 효율적인 실시간 작업 재할당 연구)

  • Bak, Na-Hyun;Shin, Jae-Young
    • Proceedings of the Korean Institute of Navigation and Port Research Conference
    • /
    • 2012.10a
    • /
    • pp.82-83
    • /
    • 2012
  • To achieve higher customer service, the operational efficiency of container terminal is important. So, to improve loading/unloading speed and reduce the time spending of berth, efficient ship planning is performed in advance. But, the condition of container terminal can be changed frequently and the pre-planning is not enough to operate container terminal sufficiently, so that we need real-time planning. This paper aims to define the problem of job reallocation of quay crane to minimize total operation time, and find the solution.

  • PDF

Development of the Web-based Interview Site With Video Conference (화상을 활용한 온라인면접 웹사이트 구축에 관한 연구)

  • 염인순;이창호
    • Journal of the Korea Safety Management & Science
    • /
    • v.3 no.3
    • /
    • pp.143-149
    • /
    • 2001
  • With the development of computer technologies and the increase of internet users, Internet plays an important role in job-recruitment business. However, almost of the sites which provide job offering and job hunting service, merely have the function such as simple inquiry for information corresponding to job offering and job hunting. In this research, the online website whose task is supporting whole procedure of recruitment for real enterprises is constructed to apply the interview procedures of the real enterprises through internet. We utilize the video conference to increase the reality for interview. Also it has functions such as presentation of application form and resume and self introduction using the video conference. Through the online recruitment website we can expect time and cost saving for recruitment for both sides of enterprise to hunt the person and people to seek the job.

  • PDF