Staffing Network Operation Center

Sumber: https://www.inoc.com/blog/staffing-a-network-operations-center

Mengawaki NOC untuk beroperasi terus menerus—tiga shift sehari, 365 hari setahun—sulit untuk direncanakan dan dikelola, belum lagi sangat mahal bagi sebagian besar corporate/lembaga yang membutuhkan dukungan ini.

Dari perspektif perencanaan, tidak ada pedoman praktis yang dapat diikuti oleh para pemimpin TI untuk menangani segala sesuatu yang masuk ke dalam operasi semacam itu, seperti:

• mengembangkan struktur operasional dan organisasi;
• mendefinisikan peran dan tanggung jawab;
• menulis specialized job description;
• membuat jadwal kerja;
• staf pelatihan;
• mengamankan tool dan lisensi;
• mengembangkan proses;
• mengelola kualitas,

…dan semua tugas penting lainnya yang perlu dilakukan dengan serius sejak awal. Tim dibiarkan merasa berkeliling dalam kegelapan untuk menyatukan semua bagian ini—selama ini mempertaruhkan bisnis/kementrian/lembaga dan pelanggan/stakeholder mereka untuk menanggung konsekuensi ketika masalah mengakibatkan pemadaman dan downtime yang mahal. Sebagian besar bisnis/kementrian/lembaga tidak mampu melakukannya karena biaya dan risiko.

Juga tidak ada pedoman untuk apa yang terjadi setelah NOC berdiri dan dikelola: mengelolanya terus-menerus—selamanya.

Menjaga NOC yang selalu aktif adalah tindakan penyeimbangan yang rumit yang membutuhkan keahlian domain khusus, metrik yang unik, dan evaluasi berkelanjutan yang bijaksana.

Kurangnya investasi di sini dapat membebani terlalu sedikit staf dengan terlalu banyak pekerjaan, yang menyebabkan kelelahan. Investasi yang berlebihan (meng-hire terlalu banyak orang untuk sebuah masalah) dapat menjadi penguras keuangan yang sangat besar bagi bisnis/kementrian/lembaga, dan masih gagal untuk mengatasi masalah yang lebih mendasar pada operasi NOC.

Karena alasan inilah (dan banyak lainnya) perusahaan sebagian atau seluruhnya meng-outsource NOC mereka ke penyedia layanan pihak ketiga ahli yang usaha intinya adalah melakukan dukungan NOC.

Berikut adalah manfaat utama outsourcing NOC sehubungan dengan kepegawaian, sekilas:

• Beban yang berat dan konstan untuk merekrut, mempekerjakan, melatih, dan mempertahankan bakat dapat di sepenuhnya di hilangkan, sehingga waktu dan perhatian kita dapat dihabiskan di tempat lain.
• Melalui skala ekonomi, tingkat dukungan yang kita terima dapat disesuaikan dengan bisnis / kegiatan kementrian/lembaga —naik atau turun— secara terduga.
• Kita dapat memanfaatkan luasnya pengalaman dan keahlian yang tidak mungkin untuk dibeli atau dihasilkan secara internal.

Singkatnya, orang atau orang yang bertanggung jawab untuk mempekerjakan dan mengelola staf NOC 24x7 harus memahami dengan jelas apa yang mereka daftarkan di awal, dan mempertimbangkan nilai tambah dari outsourcing dengan cermat.

Apa yang di bahas di bawah ini akan membantu melakukan hal itu. Kita akan mengeksplorasi beberapa tantangan utama dalam mengatur staf NOC 24x7, sehingga kita dapat mengontekstualisasikannya untuk bisnis kita dan membuat keputusan yang paling tepat tentang bagaimana kita mendapatkan dukungan yang kita butuhkan.

Jika Anda ingin lebih banyak wawasan tentang membangun tim NOC, baca posting pendamping kami: Cara Membangun dan Mengelola Tim NOC yang Efektif

Mengembangkan Struktur Organisasi NOC

Mengawaki NOC 24x7 secara efektif dimulai dengan struktur yang terorganisir dengan baik. Sebelum kita dapat menyatukan struktur organisasi berbasis tim tradisional, kita harus membuat struktur operasional yang menentukan cara kerja tim tersebut. Ini harus langsung menginformasikan siapa yang kita butuhkan di NOC.

Struktur pendukung NOC berjenjang dan antrian alur kerja yang akan sajikan di bawah ini adalah titik awal yang baik untuk menentukan tingkat keterampilan yang dibutuhkan staf NOC.

Gambar 1 mengilustrasikan struktur pendukung NOC berjenjang yang terorganisir dengan baik, yang utamanya adalah tim Tingkat 1 yang menggunakan alat pemantauan dan berinteraksi dengan meja bantuan pengguna akhir, teknisi Tingkat 2 dan 3, dan pihak ketiga.

Panah mewakili arus informasi antara berbagai entitas dalam kerangka proses yang te;ah terdefinisi.

Dengan menggunakan struktur ini, banyak NOC dapat secara efektif menyelesaikan 65 hingga 75% insiden di Tingkat 1.

Struktur ini memungkinkan kelompok pendukung untuk menangani event, permintaan layanan, dan insiden pada tier yang sesuai, dengan lebih cepat mencapai resolusi.

Workflow adalah berikutnya dalam perencanaan, yang akan membantu kita membuat struktur organisasi yang terinformasi. Workflow yang terdefinisi dengan baik juga membantu tim menghindari kesalahan-kesalahan fatal dengan merencanakan apa yang akan terjadi dan kapan. Sering kali, masalah harus diprioritaskan dan diatur ke dalam satu set antrian, yang masing-masing dapat ditangani oleh kelompok yang sesuai. Variabel seperti SLA, teknologi, dan tingkat keterampilan teknisi memainkan peran penting dalam menentukan alur kerja.

Gambar 2 menunjukkan bagaimana satu set masalah dapat dipecah menjadi antrian dan ditugaskan ke grup berdasarkan keahlian.

Contoh Antrian Workflow

Sekarang kita akhirnya bisa menyatukan struktur organisasi tim itu.

Gambar 3 di bawah menunjukkan contoh struktur NOC berbasis keterampilan yang dapat disesuaikan tidak hanya untuk mendukung persyaratan NOC 24x7, tetapi juga untuk menyediakan rencana pertumbuhan bagi karyawan, yang penting untuk retensi.

Ingatlah bahwa ini hanyalah salah satu pendekatan untuk menyusun tim NOC. Kita mungkin akan mengadopsi pola yang berbeda tergantung pada faktor yang ada.

Karena kita fokus pada staffing / kepegawaian secara khusus di sini, kita tidak akan menyelami semua proses yang kemudian perlu dikembangkan badan/institusi/perusahaan agar NOC mereka benar-benar beroperasi dengan baik secara konsisten. Tapi inilah daftar singkat dari apa yang kita harapkan perlu direncanakan jika kita berencana untuk melakukan operasi sendiri:

• Escalation
• QA/QC
• Disaster recovery/continuity
• Onboarding
• Training
• Continuous development
• Workflow management
• Incident management
• Problem management
• Knowledge management
• Access/credential management

Hiring

Once there’s an operational and staffing blueprint for the NOC, and the position descriptions are created, your HR team should be armed with clear directives for hiring and retention. This can be a major challenge that itself motivates companies to outsource to a team that lives and breathes NOC support and can take this burden off their shoulders. It’s no small task, especially in today’s labor market.

Keep in mind that the absolute minimum personnel requirement of a 24x7 NOC is 10 to 12 people.

Most NOCs need more.

Personalia

Setelah ada blue print operasional dan staf untuk NOC, dan deskripsi posisi dibuat, tim SDM harus dipersenjatai dengan arahan yang jelas untuk perekrutan dan retensi. Ini bisa menjadi tantangan besar yang memotivasi badan/lembaga/perusahaan untuk melakukan outsourcing ke tim yang akan menghidupi dukungan NOC. Ini bukan tugas kecil, terutama di pasar tenaga kerja saat ini.

Perlu diingat bahwa persyaratan personel minimum mutlak NOC 24x7 adalah 10 hingga 12 orang. Kebanyakan NOC membutuhkan lebih banyak.

Carefully consider the benefits your company provides for employees. For example, if your company provides 10 holidays and four weeks of PTO per employee, these hours need to be accounted for in the total headcount to ensure that your NOC runs smoothly.

The NOC team’s key responsibilities will likely encompass a series of tasks presented in the ITIL* framework, such as:

• Event Monitoring and Management
• Incident Management
• Problem Management
• Capacity Management
• Change Management

Ensuring the Right Skills in the NOC

It’s important to linger on the importance of skillsets for a moment. The modern NOC engineer needs a variety of skills to keep your network, infrastructure, and applications up and running.

Diverse technical knowledge, including knowledge of various network technologies, cloud environments, server operating systems, virtualization, storage systems, and applications, is becoming more and more essential—and harder to find—each day.

This demand for skilled human resources in a 24x7 environment can pose a considerable and often insurmountable challenge for many organizations.

In addition to these core skills, tech innovations demand new ones. Machine learning and artificial intelligence in particular pose new challenges that don’t always lend themselves to time-tested best practices. Even many seasoned NOC engineers haven’t dealt with networks becoming more “aware” of the traffic that runs through them. Developing skills that complement machine learning is just one example of the evolving challenges that those who work in the NOC will need to overcome.

Prioritizing NOC design (along with the right training programs and tools) to maximize your team’s capabilities from the start saves an incredible amount of expensive, labor-intensive work as the NOC comes to life.

Understanding the required skillset early on can help you identify the correct staff to hire and drive the selection of tools to manage the infrastructure over time. In addition, a rigorous, ongoing knowledge management and training program is important to ensure the entire NOC team is up to date on all changes made to the supported infrastructure.

(By the way, this is an excerpt from our free white paper: A Practical Guide to Running an Effective NOC. If you’re reading this, you should probably grab this guide, too.)

Again, given the nature of a 24x7 support operation, your team should be prepared to spend time recruiting, interviewing, training, and developing new people as a regular and likely ongoing part of their job.

Also keep in mind that the high-stress environment of a NOC naturally lends itself to turnover, especially at entry-level positions. The amount of work involved here is almost always higher than what you'd expect for the average engineering role. Whether frontline support staff move outside your company or move up within it, hiring is a near-constant and particularly high-stakes management challenge.

Is your HR team up for it?

Training A NOC training program should include initial onboarding as well as ongoing training. If you’re considering standing up or scaling an existing support operation yourself, do you have the time, energy, and expertise to develop and administer training?

If so, be prepared to incur some lead time here. It may take up to six months of various classes and on-the-job training before an engineer is ready to take on NOC support responsibilities depending on the tasks and technologies that await them.

After work has begun, monthly or quarterly training sessions should be scheduled to keep engineers’ skills fresh and to update the support team on new types of services, new customer requirements, and new equipment.

“Training is probably one of the more unspoken challenges but is probably the highest cost you'll take on. It takes time to train individuals depending on what level you're hiring them at. For a NOC, most people typically think Tier 1-style support, potentially Tier 2. Are you hiring more experienced individuals? Are you hiring people that are brand new to the industry? Because that's going to change your cost model as well. More experience costs more money. But if you pay less, and you go with somebody that's newer, they have to learn all of the skills and the processes and things that are unique to your company and to your network; how to interact with the various departments, the ticketing systems; understanding the ins and outs of your company as a whole. The more experienced person may only have to learn the company's specific tools since the technical piece is already there.”

— Austin Kelly, Director of Dedicated NOC and ATS, INOC

Retention A certain rate of attrition within the NOC should be taken into account based on your historical data and on industry standards. (Do you have historical data to work with here?)

Factors that affect retention rates include company culture as well as NOC organization (i.e., whether there’s a clear path for employee growth from one level to the next or to other departments within the organization).

By making these calculations, you can better plan for staffing and training needs. For example, assuming that a typical engineer works five years in your NOC (a retention rate of 80%), you’d need to hire an additional 20% of staff each year. This is one of many realizations that drive companies to outsource their NOC support to a dedicated support provider, who again, can shoulder this burden for them.

Scheduling There’s also the never-ending task of juggling schedules. Paid time off, sick days, and other personnel interruptions all need to be planned and covered to ensure support with required skill sets is never missing when it can’t be.

Some companies may try to mitigate the problem of weekend and third shift staff shortages by outsourcing only those shifts—a hybrid solution. However, this requires management of both the internal and external NOC staff and ensuring quality and consistency are maintained across the two different teams, rather than a single in-house or outsourced team handling everything. We make sure this is addressed thoughtfully in our service engagements.

Quality One of the biggest staffing considerations that fly under the radars of many teams—and isn’t taken into consideration—is quality control and quality assurance.

The quality of updates, communication, soft skills, etc., is something we see teams realize far too late despite being critical for the NOC to catch its errors and turn them into opportunities to improve.

Here are some of the quality-related tasks top-performing teams make part of their workflow:

Reviewing tickets, incidents, and escalations Reviewing chronic alarms and incidents Coordinating maintenance calendars Reporting on service and operational metrics Contractor assessments of performance Forecast planned activities Reviewing quantity and causes for NOC support activities Identifying root causes and preventive actions Reviewing escalations and open items Reviewing staffing and proficiency levels Validating assumptions for contract Identifying and mitigating risks within infrastructure and operations Reviewing previous action items and statuses Tools and Licenses The bigger your team, the more licenses and similar types of expenses you can expect to incur. If you’re planning on building a distributed team supporting you remotely, be sure to identify any additional costs that result, such as additional tools and sending individuals equipment. Also, make sure you have a strategy for access and credential management.

If a separate facility will be needed, be sure to factor these costs in as well as they can be considerable.

Utilization Metrics and Reporting Metrics are far too often conflated with KPIs and SLAs only—the measures and contractual obligations for how well the NOC is doing its job.

While both can certainly be a sore spot for many support teams, in the context of staffing a 24x7 NOC, there’s an even bigger metrics-related problem—one that is often a complete blind spot for teams: utilization metrics.

These are the metrics that reveal why the NOC is or isn’t busy at any point in time, and how staffing levels should be set accordingly. It’s, in many ways, the key to efficiency.

What's your throughput? What's your workload? How many tickets are you dealing with at certain points in time? How long do you spend working tickets? All of these questions are critical for setting and fine-tuning staffing levels based on how and when the NOC is typically utilized.

Some essential utilization metrics for a NOC include:

Labor content for each edit of a ticket — reveals how much work was devoted to the lifecycle of a ticket, which helps you determine optimal staffing levels, resulting in control over OPEX and better morale by avoiding overworking employees. Number of edits processed/performed per hour — shows how many edits in the ticket lifecycle a person can perform per hour, which, in combination with the previous metric, can help you gauge how many tickets can be resolved per hour. A heatmap of edits by time of day and day of week — tells you how many tickets you can expect at a given time on a given day of the week, which helps you determine optimal staffing levels. In our other guide, we dive into these utilization metrics further: NOC Performance Metrics: How to Measure and Optimize Your Operation

Staffing a 24x7 NOC vs. Outsourcing In most organizations, staffing a 24x7 NOC is often a needlessly high expenditure compared to strategically outsourcing support as a predictable operating expense. Given the payroll and overhead costs of building a NOC in-house, many companies that outsource with us cut their total cost of ownership in half.

A plan that doesn’t consider this opportunity might, for example, call for a staff of 12 full-time employees, when in fact, the same or likely better support could be provided through an outsourced service solution that takes full advantage of an economy of scale to provide far better service at a far lower cost.

In addition to staffing costs, the cost of acquiring, implementing, and integrating a full suite of NOC tools only further tips the scale in favor of outsourcing much of the time.

Perhaps the most apparent difference between homegrown and outsourced NOCs are the capabilities that come with an already-mature NOC operation and access to niche NOC expertise.

Between planning a NOC build, hiring a team, training that team, and aligning over the operational plan, in-house NOCs can expect 16 to 24 weeks minimum before all the parts are even in place. It can then take months or years to gain confident control over the system and bring it into a state of real operational maturity.

Turning up support with an outsourced NOC condenses all that time and effort into a number of weeks instead of years—often far more cost-effectively.

Read also: Shared vs. Dedicated NOC Support: A Quick-Guide

Final Thoughts and Next Steps For most companies, the cost and complexity of building and managing a NOC is a diversion from focusing its attention where it's needed most: innovating and growing the business.

Without an effective NOC, persistent support issues lead to expensive project delays, endless stress, and serious vulnerabilities that threaten your business.

Through the NOC as a service model, an outsourced support provider helps you take control of your infrastructure through a suite of NOC solutions designed to meet the specific needs of your technology environment and operational workflow—all while enabling you to focus internal resources on the projects that move the business forward.

Want to learn more about our approach to outsourced NOC support? Contact us to see how we can help you improve your IT service strategy and NOC support, schedule a free NOC consultation with our Solutions Engineers, or download our free white paper below.

White paper cover: A Practical Guide to Running an Effective NOCFREE WHITE PAPER

A Practical Guide to Running an Effective NOC Download our free white paper and learn how to build, optimize, and manage your NOC to maximize performance and uptime.

Download →

• Originally developed by the UK government’s Office of Government Commerce (OGC) - now known as the Cabinet Office - and currently managed and developed by AXELOS, ITIL is a framework of best practices for delivering efficient and effective support services.

Referensi

• https://www.inoc.com/blog/staffing-a-network-operations-center