Knowledgebase

Viewing articles tagged 'data'

Access Control Systems Who may enter.WHAT SYSTEMS USECards or tokens Codes Biometric verification Combinations of... Aggregate and Summary Tables Precomputing for speed and cost.WHAT THEY ARETables holding pre-aggregated results at a coarser... Aggregation and Grouping Summarising data.WHAT GROUPING DOESCollapses rows sharing values into one row per... Analysing and Interpreting: Everything That Matters, Briefly The summary.COMPARE LIKE WITH LIKEAgainst the same period last year, not just last month.... Analysing Costs Where money goes.WHAT TO CATEGORISEEvery cost, consistently.WHAT TO SEPARATEDirect costs, tied to... Analysing Customers Who your business actually serves.WHAT TO ANALYSERevenue by customer Margin by customer Frequency... Analysing Marketing and Channel Performance Where effort pays.WHAT TO MEASURE PER CHANNELEnquiries produced Customers produced Cost Cost per... Analysing Profitability by Product or Service Where money is actually made.WHAT TO CALCULATE PER ITEMRevenue Direct cost Gross profit Gross... Analysing Sales Patterns When and how business arrives.WHAT TO LOOK ATSales by month, week and day Sales by hour, for... Analysing Seasonality Understanding your year.WHY IT MATTERSMost businesses are seasonal, and many owners underestimate... Analytical Versus Transactional Databases Two different purposes.WHAT TRANSACTIONAL SYSTEMS OPTIMISE FORMany small reads and writes Strong... Analytics Engineering: Everything That Matters, Briefly The summary.THE WORK IS ENCODING BUSINESS DEFINITIONSGetting them wrong is invisible until... Analytics for Businesses Without a Data Person Realistic capability.WHAT YOU ACTUALLY NEEDRecords that are complete and consistent A spreadsheet... Automating Data Collection Reducing manual work.WHAT TO AUTOMATEExports from systems Combining routine sources Recalculating... Automation in Data Centre Operations Reducing manual work.WHAT IS AUTOMATEDMonitoring and alerting Provisioning of servers and virtual... Avoiding Spreadsheet Errors Where mistakes hide.THE MOST COMMON ERRORSRanges that do not include all the data Formulas copied... Backfilling and Reprocessing Correcting history.WHEN IT IS NEEDEDA bug is found A source is corrected A definition changes New... Batch and Streaming Approaches When data is processed.WHAT BATCH PROCESSING ISProcessing accumulated data at intervals.WHAT... Building a Data Platform Incrementally Growing capability sensibly.WHAT TO BUILD FIRSTOne pipeline, from one source, producing one... Building a Semantic Layer Definitions in one place.WHAT IT ISA layer holding metric and dimension definitions, used by... Building a Simple Data Model for Your Business Connecting your records.WHAT A DATA MODEL ISHow your records relate to each other.WHAT A SMALL... Building a Simple Reporting Spreadsheet A monthly template.WHAT IT SHOULD CONTAINA data sheet you paste new exports into Calculations... Building a Small Set of Business Metrics Your own scorecard.WHAT TO INCLUDEFive to eight figures covering income, profitability, cash,... Building a Useful Dashboard A summary people read.WHAT A DASHBOARD IS FORShowing current state at a glance.WHAT IT IS NOT... Building an Analytics Habit Making it routine.WHY A HABITOccasional analysis produces occasional insight and no trend.WHAT TO... Building Analytics Skills in Your Team Capability.WHAT TO LEARN FIRSTSpreadsheet structure and pivot summaries Conditional totals and... Building Data Products Treating data as something delivered.WHAT A DATA PRODUCT ISA dataset or service with an owner,... Building Reliable Dashboards Reports people can trust.WHAT MAKES A DASHBOARD FAILNumbers that disagree with other sources... Building Reliable Ingestion Making extraction dependable.WHAT RELIABILITY REQUIRESIdempotence: running twice produces the... Building Structure and Layout How facilities are arranged.WHAT THE MAIN AREAS AREThe data halls, containing racks Power rooms,... Cable Management Organisation that affects everything.WHY IT MATTERSPoor cabling obstructs airflow, prevents... Careers and Skills in Data Centre Operations Working in the field.WHAT ROLES EXISTFacility technicians, handling power and cooling plant Data... Change Data Capture Reading changes from the source.WHAT IT ISCapturing inserts, updates and deletes by reading the... Change Management in Facilities Controlling what is altered.WHY IT MATTERSMost facility incidents follow a change.WHAT A CHANGE... Checklist: Before You Trust an Analysis Verification.THE DATAWhere did it come from? What period does it cover? Does the total match the... Checklist: Monthly Analysis Routine An hour a month.PREPAREExport the period's data from each source Check totals against each system... Checklist: Setting Up Analysis for a New Business From the start.DECIDE WHAT TO RECORDSales, with date, customer, item, price and cost Enquiries,... Choosing a Data Centre Site Where facilities are built.WHAT MATTERS IN SITE SELECTIONPower availability and reliability... Choosing a Data Warehouse Selecting the analytical store.WHAT THE MAIN CATEGORIES ARECloud warehouses, billed by usage... Choosing Analytics Tools What to use.WHAT MOST SMALL BUSINESSES NEEDA spreadsheet, and the reporting built into systems... Choosing the Right Chart Matching form to content.FOR CHANGE OVER TIMEA line chart.Nothing else shows direction as... Choosing Which Numbers to Track Selecting metrics.THE TESTWould you act differently if this changed?If not, do not track it.WHAT... Cleaning Data Before Analysis Preparation.WHAT TO CHECK FIRSTDuplicates Missing values Obvious errors Inconsistent categories... Cohort Analysis Explained Simply Grouping by when they started.WHAT A COHORT ISCustomers grouped by when they first bought.WHY IT... Collecting and Organising Data: Everything That Matters, Briefly The summary.THE MOST VALUABLE FIELD YOU CAN ADDHow each enquiry or customer found you.It answers... Combining Batch and Streaming Architectures using both.WHAT THE TENSION ISStreaming provides freshness; batch provides... Combining Data From Different Sources Joining records.WHEN THIS ARISESSales in one system, customers in another Costs in accounting,... Common Analytical Mistakes Errors that produce wrong conclusions.CONFUSING CORRELATION WITH CAUSETwo things moving together... Common Misconceptions About Data Centres What people get wrong."THE CLOUD IS NOT PHYSICAL"It is buildings full of equipment, with power... Common Reporting Mistakes What goes wrong.TOO MUCHEvery number, so nothing stands out.NO COMPARISONA figure with nothing to... Common Table Expressions and Query Structure Writing readable SQL.WHAT A COMMON TABLE EXPRESSION ISA named subquery defined before the main... Comparing Periods Properly Like against like.WHAT TO COMPAREThis month against last month This month against the same month... Console Access and Remote Control Reaching equipment directly.WHAT CONSOLE ACCESS PROVIDESThe equipment's own display and keyboard,... Cooling and Environment: Everything That Matters, Briefly The summary.EQUIPMENT CONVERTS ESSENTIALLY ALL ITS ELECTRICITY INTO HEATA rack consuming several... Cooling Efficiency and Operating Practice Reducing waste.WHAT WASTES COOLINGHot and cold air mixing Missing blanking panels Perforated... Cooling Redundancy and Failure When cooling stops.WHAT HAPPENSTemperature rises quickly, and in a dense hall, very quickly.WHAT... Cost Management in Data Platforms Controlling spend.WHAT DRIVES COSTCompute for queries and processing Storage Data transfer... Data Analytics and Your Obligations Handling data responsibly.WHAT ANALYSIS FREQUENTLY INVOLVESPersonal data about customers and... Data Analytics: Everything That Matters, Briefly The whole category in one page.START WITH A QUESTION YOU WOULD ACT ONIf the answer changes... Data Centre Operations Day-to-day running.WHAT OPERATIONS COVERSMonitoring conditions and systems Responding to alarms... Data Centre Technology: Everything That Matters, Briefly The whole category in one page.WHAT YOU ARE ACTUALLY BUYING WITH HOSTINGThe facility as much as... Data Centres and Intensive Computing Workloads Changing demands.WHAT HAS CHANGEDCertain workloads consume far more power per rack than... Data Contracts and Producer Agreements Agreeing what data will be.WHAT A DATA CONTRACT ISAn explicit agreement between a producer and... Data Engineering for AI Applications Pipelines feeding language models.WHAT RETRIEVAL-BASED APPLICATIONS REQUIREA corpus of documents... Data Engineering Skills and Tools What the work requires.WHAT IS ESSENTIALSQL, thoroughly One programming language, usually Python... Data for Machine Learning: Everything That Matters, Briefly The summary.LEAKAGE IS THE MOST DAMAGING ERROR AVAILABLEThe model tests excellently and fails... Data Governance in Practice Controlling data as an organisational asset.WHAT IT COVERSWhat data exists Who owns it Who may... Data Ingestion Patterns How data arrives.WHAT THE PATTERNS AREFull extraction: reading everything each timeIncremental... Data Lakes and Lakehouse Formats Storage-based platforms.WHAT A DATA LAKE ISObject storage holding files, in any format, at low... Data Modelling Fundamentals Representing a business in tables.WHAT A DATA MODEL ISA representation of entities, their... Data Modelling: Everything That Matters, Briefly The summary.DECLARING THE GRAIN IS THE MOST IMPORTANT MODELLING DECISIONWhat one row represents.... Data Quality and Why It Decides Everything The foundation.THE PRINCIPLEAnalysis is only as good as the records beneath it.WHAT POOR DATA... Data Quality Dimensions What quality actually means.WHAT THE DIMENSIONS AREAccuracy: values reflect realityCompleteness:... Databases and Storage: Everything That Matters, Briefly The summary.ROW STORAGE SUITS TRANSACTIONS; COLUMN STORAGE SUITS ANALYSISWhich is why a read... Deciding What to Record Collection with purpose.THE PRINCIPLERecord what you will use.WHAT TO RECORD FOR EVERY SALEWhat... Decommissioning Equipment Removing things properly.WHAT TO ESTABLISH FIRSTWhat the equipment does What depends on it... Defining Metrics Consistently One definition, everywhere.WHAT THE PROBLEM ISThe same metric calculated differently in different... Delivery Guarantees What a system promises about each record.WHAT AT-MOST-ONCE MEANSEach record processed zero or one... Designing a Data Warehouse Structuring the whole.WHAT LAYERS TO BUILDRaw: data as received, unmodifiedStaging or cleaned:... Designing Event Schemas Structuring what flows.WHAT EVERY EVENT SHOULD CARRYA unique identifier The event type When it... Designing for Business Intelligence Tools Modelling for how tools query.WHAT REPORTING TOOLS DOGenerate queries from user interactions with... Designing Pipelines for Failure Assuming things break.WHAT WILL FAILSources unavailable Credentials expiring Schemas changing... Dimensional Modelling Facts and dimensions.WHAT A FACT TABLE HOLDSMeasurements of events: sales, payments, page views,... Disaster Recovery and Business Continuity Surviving serious events.WHAT FACILITIES PROTECT AGAINSTComponent failure Power loss Cooling... Distributed Processing Frameworks Processing across machines.WHAT THEY PROVIDESplitting work across machines Handling of machine... Documenting Your Analysis So it can be repeated and trusted.WHAT TO RECORDWhere the data came from, and for what period... Edge Computing and Distributed Facilities Bringing computing closer.WHAT EDGE COMPUTING MEANSProcessing closer to where data is generated... Electrical Safety in Facilities Working safely with power.WHY IT MATTERSData centre electrical systems carry enough energy to... Emergency Procedures and Safety When something serious happens.WHAT PROCEDURES SHOULD EXISTFire evacuation Medical emergency... Enabling Self-Service Analytics Letting others answer their own questions.WHAT IT REQUIRESData modelled so it is understandable... Environmental and Sustainability Considerations Energy and impact.WHAT DATA CENTRES CONSUMESubstantial electricity, continuously Water, in some... Environmental Monitoring Watching conditions.WHAT TO MONITORTemperature, at intakes and at several heights Humidity... Essential Spreadsheet Functions for Business Analysis The few worth knowing.SUM AND AVERAGETotals and means. Obvious and constantly used.CONDITIONAL... Establishing Definitions Everyone Agrees On Shared meaning.THE PROBLEMTwo people reporting the same metric, differently.WHAT CAUSES ITNo... Evaluating Local and International Hosting Where to place your services.WHAT LOCAL HOSTING PROVIDESLower latency for local visitors Billing... Extracting From Application Interfaces Pulling data from services.WHAT TO ESTABLISH FIRSTRate limits Pagination approach Whether... Facility Design: Everything That Matters, Briefly The summary.SITE SELECTION IS DECIDED BY POWER AND CONNECTIVITYEverything else is compensation. A... Feature Engineering Pipelines Building model inputs.WHAT A FEATURE ISA value derived from data, used as model input.WHAT COMMON... Feature Stores Managing features centrally.WHAT ONE ISA system storing feature definitions and values, serving... File Formats for Data How data is physically stored.WHAT TEXT FORMATS PROVIDEHuman readability Universal supportWHAT... Fire Detection and Suppression Protecting against fire.WHY IT MATTERSA fire in a facility can destroy equipment and data... Forecasting and Planning: Everything That Matters, Briefly The summary.THE SIMPLEST METHOD THAT WORKSLast year's same period, adjusted for known changes.For... Forecasting Cash The forecast that matters most.WHY CASH RATHER THAN PROFITA profitable business can run out of... Forecasting Demand and Stock Planning what you need.WHAT TO FORECASTUnits expected to sell, by period, by item.WHAT TO BASE IT... Forecasting Without Complicated Methods Practical prediction.WHAT A FORECAST ISAn estimate of what is likely, based on what has... Foundations: Everything That Matters, Briefly The summary.A PIPELINE THAT RUNS SUCCESSFULLY AND PRODUCES WRONG NUMBERS IS WORSE THAN ONE THAT... Geospatial Data Location in data pipelines.WHAT THE TYPES AREPoints: a single locationLines: routes and... Getting Data Out of Your Systems Exports.WHERE DATA LIVESAccounting software Shop platform Payment gateway Website analytics... Governing Data Access for Analysis Who can see what.WHAT ANALYSIS DATA CONTAINSFrequently personal data: customers, staff,... Graph and Relationship Data Modelling connections.WHAT IT REPRESENTSEntities and the relationships between them.WHERE IT... Handling Change in Dimensions When descriptive data changes.WHAT THE PROBLEM ISA customer moves city. Do past sales belong to... Handling Currency and Exchange Rates Money across currencies.WHY IT MATTERS HERERevenue and costs frequently span local and foreign... Handling Files and Delimited Data The most common and most troublesome source.WHAT GOES WRONG WITH DELIMITED FILESSeparators... Handling Personal Data in Pipelines Privacy in data engineering.WHAT TO ESTABLISH FIRSTWhat personal data you hold, and why.WHAT... Handling Schema Changes When sources change shape.WHAT CHANGES OCCURColumns added Columns removed Columns renamed Types... Hardware Lifecycle and Refresh Equipment over time.WHAT HAPPENS AS EQUIPMENT AGESFailure rates rise Support and parts become... Hot and Cold Aisle Arrangement Rack orientation.WHAT THE ARRANGEMENT ISRacks placed in rows facing each other alternately.WHAT... How Data Centre Cooling Works The basic principle.WHAT COOLING DOESRemoves heat from the air, and rejects it outside the... How Much Data Is Enough Sample size, practically.THE QUESTIONHow many records before a conclusion is safe?THE HONEST... How Power Reaches Equipment The electrical path.THE SEQUENCESupply enters the building Switchgear distributes it... Implementing Data Quality Testing Catching problems automatically.WHAT TO TEST AT INGESTIONSchema matches expectation Row counts... Incremental Processing Processing only what changed.WHY IT MATTERSReprocessing everything becomes infeasible as volume... Ingestion and Integration: Everything That Matters, Briefly The summary.DELETES BREAK INCREMENTAL EXTRACTIONA deleted row simply stops appearing. Use change... Inside the Data Hall: Everything That Matters, Briefly The summary.PLAN THE RACK BEFORE INSTALLING ANYTHINGHeavier equipment lower, nothing obstructing... Installing Equipment in a Rack Physical installation.WHAT TO PREPAREThe rails or mounting hardware The correct tools Labels An... Integrating Data From Several Sources Combining systems that disagree.WHAT THE PROBLEMS AREThe same entity identified differently in... Inventory and Asset Management Knowing what you have.WHAT TO RECORD FOR EVERY ITEMWhat it is Serial numberWhere it is: room,... Investigating Why Something Changed Finding causes.WHERE TO STARTSegment it.A fall overall is usually a fall in one part.WHAT TO... Joins in Practice Combining tables correctly.WHAT THE TYPES DOInner: rows matching in bothLeft: every row from the... Keeping Analysis Honest Integrity.THE RISKAnalysis produced to support a decision already made.WHAT THAT LOOKS... Keeping Data Consistent The discipline that makes analysis possible.WHAT INCONSISTENCY LOOKS LIKEThe same customer... Labelling Standards Making everything identifiable.WHAT TO LABELEvery rack Every unit position Every device Every... Layers of Physical Security Defence in depth.WHAT THE LAYERS AREThe perimeter: fencing, barriers, lightingThe building... Leading and Lagging Indicators Two kinds of measure.LAGGING INDICATORSResults after the fact: revenue, profit, customers... Liquid Cooling Cooling beyond air.WHAT IT ISRemoving heat using liquid in direct or close contact with... Maintenance and Testing Schedules Preventive work.WHAT REQUIRES REGULAR MAINTENANCEGenerators Uninterruptible supplies and their... Making Charts Readable Presentation that communicates.WHAT TO INCLUDEA title stating the finding, not the subject... Managing Hot Spots Uneven temperature.WHAT A HOT SPOT ISA location where intake temperature is significantly higher... Managing Network Security in a Facility Protecting connectivity.WHAT TO SEPARATECustomer networks from each other Management networks... Managing Pipeline Code Engineering practice for data.WHAT TO VERSION CONTROLTransformation code Pipeline definitions... Managing Power Quality Clean electricity.WHAT POWER PROBLEMS OCCURSurges Dips and brownouts Complete interruptions... Managing Visitors and Contractors People who are not staff.WHAT TO REQUIREAdvance notification Identity verification on arrival A... Measuring Acquisition Cost What it costs to win a customer.WHAT TO CALCULATEAll money spent attracting customers in a period... Measuring Customer Value What a customer is worth.WHAT TO CALCULATEAverage purchase value How often they buy How long they... Measuring Marketing and Sales Conversion From enquiry to customer.WHAT TO MEASUREEnquiries received Quotes or proposals issued Customers... Measuring Operational Efficiency Doing the work.WHAT TO MEASURETime taken per job or unit Quoted versus actual Rework and errors... Measuring Retention and Churn Customers staying and leaving.WHAT RETENTION ISThe proportion of customers who continue... Measuring What Matters: Everything That Matters, Briefly The summary.THE TEST FOR ANY METRICWould you act differently if it changed? If not, do not track... Migrating Between Facilities Moving your hosting.WHAT TO ESTABLISH FIRSTWhat must move What depends on what How much downtime... Modelling Customer Data A single view of a customer.WHAT IT MEANSOne record per customer, combining information from... Modelling Product and Web Events Behavioural data.WHAT AN EVENT REPRESENTSAn action taken, at a time, by someone.WHAT EVERY EVENT... Modelling Slowly Changing Facts and Events Handling corrections and late data.WHAT LATE-ARRIVING DATA ISRecords for a period that arrive... Modelling Subscription and Billing Data Recurring revenue.WHAT THE ENTITIES ARECustomers Subscriptions Plans and prices Invoices Payments... Modular and Prefabricated Facilities Building differently.WHAT THEY AREFacilities assembled from prefabricated units, rather than... Monitoring Data Pipelines Knowing they worked.WHAT TO MONITORWhether each run completed Duration, against normal Row counts... Naming and Documentation Standards Making a warehouse usable.WHY IT MATTERSA warehouse nobody can navigate is used by nobody,... Network Architecture in a Data Centre How equipment connects.WHAT THE TRADITIONAL ARRANGEMENT ISSwitches in each rack, connecting... Network Infrastructure: Everything That Matters, Briefly The summary.REDUNDANCY MUST BE GENUINELY DIVERSETwo cables in one tray, two connections to one... Network Monitoring in the Data Centre Watching the infrastructure.WHAT TO MONITORLink status and utilisation Error and discard rates... Network Redundancy and Resilience Surviving failures.WHAT CAN FAILA cable A switch A switch port An upstream provider A fibre route... NoSQL Stores in Data Engineering Non-relational systems.WHAT THE CATEGORIES AREKey-value: fast lookup by identifierDocument:... Operating Streaming Systems Running them reliably.WHAT TO MONITORConsumer lag, per partition Throughput, in and out Error and... Operations and Standards: Everything That Matters, Briefly The summary.BACKUP SYSTEMS FAIL MORE OFTEN THAN ANYONE EXPECTS, BECAUSE THEY ARE RARELY... Orchestration and Operations: Everything That Matters, Briefly The summary.NEVER WRITE A TASK THAT IMPLICITLY USES THE CURRENT DATEIt cannot be backfilled, and... Out-of-Band Management Access independent of the network.WHAT IT ISA separate path to manage equipment, independent of... Partitioning and Data Layout Organising for query efficiency.WHAT PARTITIONING DOESDivides data so queries read only relevant... Physical Security: Everything That Matters, Briefly The summary.SECURITY IS LAYEREDPerimeter, building, zone, hall, rack. Each must be defeated in... Planning a Rack Layout Deciding what goes where.WHAT TO PLAN BEFORE INSTALLING ANYTHINGWhich equipment occupies which... Planning for Capacity and Growth Designing for the future.WHAT CONSTRAINS GROWTHPower, first Cooling, second Space, lastWHY THAT... Planning With Scenarios Preparing for different outcomes.WHAT A SCENARIO ISA coherent set of assumptions about the... Power Distribution in Racks From the room to the device.WHAT A POWER DISTRIBUTION UNIT ISA device within the rack providing... Power Efficiency and Measurement How much is wasted.WHAT THE STANDARD MEASURE ISTotal facility power divided by power delivered to... Power Systems: Everything That Matters, Briefly The summary.THE POWER PATH HAS MANY STAGESSupply, switchgear, uninterruptible supplies,... Preparing Training Data Data for model building.WHAT DISTINGUISHES IT FROM ANALYTICAL DATAIt must reflect what will be... Presenting and Reporting: Everything That Matters, Briefly The summary.CHOOSE THE FORM BY THE MESSAGELine for change over time, bar for comparing... Presenting Data Honestly Avoiding misleading.WHAT MISLEADS, EVEN UNINTENTIONALLYValue axes not starting at zero, on bar... Processing Data With Python Practical transformation code.WHAT THE COMMON LIBRARIES AREA long-established tabular library... Processors and Memory in Servers Computing capacity.WHAT PROCESSORS PROVIDECores, each able to execute work independently Clock... Protecting Against Traffic Floods Volumetric attacks.WHAT THEY AREAttempts to overwhelm a service or a connection with traffic.WHY... Quality, Governance and Practice: Everything That Matters, Briefly The summary.MOST DATA INCIDENTS ORIGINATE UPSTREAM, IN CHANGES NOBODY COMMUNICATEDData contracts... Query Performance and Optimisation Making queries fast.WHAT TO DO FIRSTRead the query plan.WHAT TO LOOK FORFull scans where an index... Questions to Ask a Hosting or Facility Provider Before committing.ABOUT THE FACILITYWhere is it, precisely? Who operates it? What certifications... Raised Floors and Containment Airflow management.WHAT A RAISED FLOOR DOESCreates a plenum beneath the floor for cold air, power... Reference: Analytics Terms Explained Vocabulary.METRICA figure being tracked.KPIA metric considered important enough to steer... Reference: Questions Your Data Should Answer A starting list.MONEYWhich products or services produce the most profit? Which lose money? Is our... Remote Hands and On-Site Support Physical work you cannot do yourself.WHAT REMOTE HANDS MEANSFacility staff performing physical... Reporting Regularly Cadence.WHAT TO REPORT WEEKLYLeading indicators only: enquiries, quotes, bookings, cash.Short,... Reporting to Different Audiences Adapting the message.FOR AN OWNER OR MANAGERThe finding, the implication, the decision... Reproducibility in Data Science Work Making results repeatable.WHAT MUST BE REPRODUCIBLEThe dataset The transformations The code The... Scheduling and Dependencies When things run.WHAT TIME-BASED SCHEDULING ASSUMESThat inputs are ready by the scheduled time.WHY... Securing Your Own Equipment in a Shared Facility Customer responsibilities.WHAT REMAINS YOURSYour equipment, its configuration, and its physical... Segmenting Your Data Breaking totals apart.WHY SEGMENTA total hides everything interesting.WHAT TO SEGMENT BYProduct... Server Storage Options Disks and arrays.WHAT TYPES EXISTMechanical drives: high capacity, lower cost, slowerSolid-state... Servers and Storage: Everything That Matters, Briefly The summary.WHAT MAKES SERVER HARDWARE DIFFERENTContinuous operation, redundant components,... Service Level Agreements for Facilities What is committed.WHAT IS TYPICALLY COMMITTEDPower availability Environmental conditions within a... Serving Data to Consumers Making it usable.WHO THE CONSUMERS AREAnalysts, querying directly Reporting tools Applications... Serving Data to Models Delivering inputs at prediction time.WHAT THE PATTERNS AREBatch prediction: scoring everything on... Setting Targets That Mean Something Goals attached to numbers.WHAT A USEFUL TARGET ISSpecific Based on your own history Achievable... Setting Up a Spreadsheet for Analysis Structure first.WHAT TO SEPARATERaw data, on its own sheet Calculations, on another Output and... Sharing Spreadsheets Safely Collaboration and control.THE PROBLEMSeveral people editing, versions multiplying, nobody knowing... Spares and Consumables What to keep on site.WHAT TO CONSIDER KEEPINGSpare drives, matching those in use Power supplies,... Spotting Trends Rather Than Noise Signal and variation.THE PROBLEMEvery figure varies. Most variation means nothing.WHAT NOISE... Spreadsheet Habits Worth Adopting Discipline that pays.ALWAYS WORK ON A COPYNever on the only version of the source.CHECK A TOTAL... Spreadsheets: Everything That Matters, Briefly The summary.THE STRUCTURE THAT MAKES EVERYTHING WORKRaw data on one sheet, calculations on... SQL Across Different Engines Dialect differences.WHAT IS STANDARDCore query structure, joins, aggregation, and much else.WHAT... SQL Beyond the Basics What separates competent from capable.WHAT MOST PEOPLE KNOWSelecting, filtering, sorting, and... SQL: Everything That Matters, Briefly The summary.UNDERSTAND THE EXECUTION ORDERJoin, filter, group, filter groups, compute, sort,... Standards and Certifications Independent assessment.WHAT CERTIFICATIONS ADDRESSInformation security management Service... Starting With a Question, Not a Tool The right sequence.WHAT PEOPLE DOBuy a tool, connect data, build a dashboard, and look at it... Statistics Without the Mathematics: Everything That Matters, Briefly The summary.SMALL NUMBERS SHOW PATTERNS THAT ARE COINCIDENCETen records will always show... Storage Cost and Lifecycle Managing what accumulates.WHAT DRIVES COSTVolume stored Access frequency and pattern Data... Storage Systems and Shared Storage Storage beyond the server.WHAT SHARED STORAGE ISStorage accessible by several servers, rather... Storing and Organising Your Analysis Files File discipline.WHAT ACCUMULATESExports Working copies Versions Final outputsWHAT GOES... Storytelling With Business Data Making findings land.WHY IT MATTERSNumbers alone rarely change behaviour.WHAT A FINDING... Stream Processing Concepts Computing over flows.WHAT STATELESS PROCESSING ISEach record processed independently: filtering,... Streaming Platforms and Tools The available systems.WHAT LOG-BASED PLATFORMS PROVIDEDurable, ordered, replayable streams with... Streaming: Everything That Matters, Briefly The summary.ORDERING IS GUARANTEED WITHIN A PARTITION, NOT A TOPICEvents for one entity must... Structural and Environmental Protection Physical resilience.WHAT FACILITIES PROTECT AGAINSTWater ingress Dust Pests Vibration... Structured Cabling Physical connectivity.WHAT IT ISA planned, documented cabling system rather than ad hoc... Structuring Data So It Can Be Analysed Format matters.THE RULEOne row per record, one column per attribute, one value per cell.WHAT THAT... Structuring Transformation Layers Organising models.WHAT LAYERS TO USESources: declared references to raw tablesStaging: one model... Summarising Data With Pivot Summaries The most useful feature.WHAT THEY DOGroup records and summarise them without writing... Surveillance and Monitoring Watching the facility.WHAT IS TYPICALLY MONITOREDPerimeter and entrances Corridors and access... Temperature and Humidity Ranges What conditions equipment needs.WHAT MATTERSTemperature at equipment intakes, not room... Testing Data Transformations Verifying correctness.WHY IT DIFFERS FROM SOFTWARE TESTINGThe code may be correct while the data... Testing Whether a Change Actually Worked Knowing whether something worked.WHAT TO DO BEFORE CHANGING ANYTHINGRecord the current... The Data Engineering Lifecycle How work proceeds.WHAT THE STAGES AREUnderstanding the requirement Identifying and assessing... The Data Platform Landscape Components and how they fit.WHAT THE LAYERS ARESources: applications, databases, files, external... The Difference Between Data, Information and Insight Why the distinction matters.DATARaw records. Transactions, timestamps, entries.Meaningless on its... The Growth of Data Centres in Africa Regional development.WHAT HAS CHANGEDSubstantial investment in facilities across the continent,... The Main Types of Data Centre Ownership and purpose.ENTERPRISEBuilt and operated by one organisation for its own... Tools and Governance: Everything That Matters, Briefly The summary.WHAT MOST SMALL BUSINESSES NEEDA spreadsheet and the reporting already built into... Transformation Frameworks Managing transformation as code.WHAT THEY PROVIDETransformations defined as select statements,... Transformation: Everything That Matters, Briefly The summary.LOAD RAW, THEN TRANSFORM IN PLACETransformation is far easier to test, version and... Trends and Wider Context: Everything That Matters, Briefly The summary.HYPERSCALE FACILITIES ASSUME FAILURE RATHER THAN PREVENTING ITThey build redundancy... Turning Analysis Into Decisions Closing the loop.THE PROBLEMAnalysis produced, read, and nothing changes.WHAT MAKES ANALYSIS... Understanding Availability and Uptime What continuous operation means.WHAT UPTIME MEASURESThe proportion of time a service is... Understanding Averages and What They Hide Beyond the mean.WHAT AN AVERAGE DOESSummarises many values into one.WHAT IT HIDESThe spread... Understanding Bandwidth and Capacity Network throughput.WHAT BANDWIDTH ISThe maximum rate a link can carry.WHAT COMMITTED CAPACITY... Understanding Bias in Data Systematic error.WHAT BIAS ISA consistent distortion, rather than random error.WHY IT IS WORSE... Understanding Colocation Renting space for your own equipment.WHAT YOU RENTRack space, measured in units Power, at a... Understanding Confidence and Uncertainty Knowing how sure to be.WHAT UNCERTAINTY MEANSThe finding could be wrong, and by how much.WHAT... Understanding Correlation and Cause The distinction that matters most.WHAT CORRELATION MEANSTwo things moving together.WHAT IT DOES... Understanding Cross-Connects Connections between parties.WHAT A CROSS-CONNECT ISA physical cable linking your equipment to... Understanding Data Analytics: Everything That Matters, Briefly The summary.WHAT IT ACTUALLY ISUsing the information your business already produces to answer... Understanding Data at Different Scales Matching approach to volume.WHAT SMALL DATA MEANSFits comfortably in memory on one machine.WHAT... Understanding Data Centres: Everything That Matters, Briefly The summary.WHAT A DATA CENTRE PROVIDESContinuous power, controlled cooling, multiple network... Understanding Data Residency Where data is held.WHAT IT MEANSThe physical location where data is stored and processed.WHY IT... Understanding Data Sources Where data comes from.WHAT THE COMMON SOURCES AREApplication databases Event streams from... Understanding Data Vault and Alternative Approaches Other modelling methods.WHAT DATA VAULT ISA modelling approach separating business keys,... Understanding Distributions How values are spread.WHAT A DISTRIBUTION ISHow values are spread across a range.WHY IT... Understanding Drive Failure and Replacement Storage reliability.WHAT TO EXPECTDrives fail. In any substantial population, regularly.WHAT... Understanding Energy Costs What power costs to run.WHAT DRIVES COSTEnergy consumed by equipment Energy consumed by cooling... Understanding ETL and ELT Where transformation happens.WHAT ETL ISExtract, transform, then load the transformed result.WHAT... Understanding Event Streaming Data as a continuous flow.WHAT AN EVENT ISA record that something happened, at a point in... Understanding Generators Extended backup power.WHAT THEY DOSupply the facility when mains power is unavailable, for as... Understanding Hosting Costs What you pay for.WHAT DRIVES COST IN A FACILITYPower, primarily Cooling, which follows power... Understanding Hyperscale Facilities Very large data centres.WHAT THEY AREEnormous facilities operated by the largest cloud and... Understanding Internet Exchange Points Where networks meet.WHAT AN EXCHANGE POINT ISA facility where networks connect directly to each... Understanding Latency and Location Why the facility's location matters.WHAT LATENCY ISThe time for data to travel there and back.WHY... Understanding Outliers Unusual values.WHAT AN OUTLIER ISA value far from the rest.WHAT IT MIGHT BEAn error A genuine... Understanding Percentages and Rates Getting proportions right.WHAT A PERCENTAGE NEEDSA base. Percentage of what.WHAT GOES WRONG... Understanding Pipeline Orchestration Coordinating work.WHAT AN ORCHESTRATOR DOESDefines tasks and their dependencies Schedules them... Understanding Power Capacity and Density How much a rack can take.WHAT DENSITY MEANSPower consumed per rack, expressed in kilowatts.WHAT... Understanding Racks and Units How equipment is arranged.WHAT A RACK ISA standardised frame holding equipment.WHAT A UNIT ISThe... Understanding Redundancy Having more than one of things.WHAT REDUNDANCY MEANSA second of something, so a failure does not... Understanding Revenue, Profit and Margin The figures people confuse.REVENUETotal sales value.The number people quote and the least useful... Understanding Server Hardware What a server is.WHAT MAKES IT DIFFERENT FROM A DESKTOPDesigned to run continuously Built for... Understanding Submarine Cables and Connectivity How international traffic travels.WHAT CARRIES INTERCONTINENTAL TRAFFICSubmarine fibre optic... Understanding Tier Classifications Facility ratings.WHAT THEY DESCRIBEThe level of redundancy and maintainability in a facility's... Understanding Uncertainty in Forecasts Being honest about what you do not know.WHY FORECASTS ARE WRONGThey assume the future resembles... Understanding Uninterruptible Power Supplies Bridging the gap.WHAT THEY DOProvide power instantly when the supply fails, until generators... Understanding Variation Why numbers move.WHAT VARIATION ISNatural fluctuation in any measure.WHY IT MATTERSMost movement... Understanding Virtualisation in the Data Centre Several systems on one machine.WHAT IT ISRunning several independent systems on one physical... Using AI Tools for Analysis Care and limits.WHAT THEY CAN HELP WITHExplaining a function or method Suggesting how to... Using Data for Capacity and Staffing Matching resources to demand.WHAT TO ANALYSEDemand by day, week and hour Time taken per unit of... Using Data for Pricing Decisions Evidence behind prices.WHAT TO ESTABLISH FIRSTActual cost per item or service, including your... Using Ranking and Contribution Analysis Finding what matters most.WHAT IT ISRanking items by contribution, and seeing how much comes from... Vector Storage and Retrieval Storing embeddings.WHAT A VECTOR STORE DOESHolds embeddings and finds the most similar to a query... Visiting a Data Centre What to expect and prepare.WHAT TO ARRANGE IN ADVANCEAuthorisation Notification of who is... What a Customer Is Responsible For The division of duties.WHAT THE FACILITY PROVIDESPower, cooling, connectivity, physical security,... What a Data Centre Actually Is The building behind the cloud.WHAT IT ISA facility housing computing equipment, with the power,... What Analytics Engineering Is The role between engineering and analysis.WHAT IT ISTransforming raw data into modelled, tested,... What Data Analytics Means for a Small Business Scope and purpose.WHAT IT ISUsing the information your business already produces to answer... What Data Engineering Is The discipline and its purpose.WHAT IT ISBuilding and operating the systems that collect, move,... What Data Your Business Already Produces Taking stock.FROM SALESWhat was sold, when, to whom, at what price Discounts given Returns and... What You Cannot Learn From Data Honest limits.WHY SOMETHING HAPPENEDData shows what happened. It rarely shows why.WHAT CUSTOMERS... When to Move Beyond Spreadsheets Recognising the limit.WHAT SPREADSHEETS DO WELLAd hoc analysis Modest volumes One person working... Why Businesses Analyse Their Data The reasons that matter.TO ANSWER QUESTIONSWhich of my products is most profitable? Why did sales... Why Businesses Use Data Centres The alternative to a server in the office.WHAT A SERVER IN AN OFFICE FACESPower interruptions... Why Cooling Matters Heat as the primary constraint.WHAT EQUIPMENT DOESConverts essentially all the electricity it... Window Functions Calculations across related rows.WHAT THEY DOCompute a value for each row, using a set of related... Working as a Data Engineer The role in practice.WHAT THE WORK ACTUALLY ISBuilding and maintaining pipelines Investigating... Working Safely in a Data Hall Personal safety and care.WHAT THE ENVIRONMENT IS LIKELoud, cold in places and hot in others, with... Working With Business Teams Getting definitions right.WHAT THE COMMON FAILURE ISBuilding what was asked for rather than what... Working With Dates and Periods Time in spreadsheets.WHY DATES CAUSE PROBLEMSThey are stored as numbers and displayed as text,... Working With Dates and Times in SQL The commonest source of subtle errors.WHAT TO ESTABLISH FIRSTWhat time zone data is stored in,... Working With Unstructured Data Text, documents, images and audio.WHAT MAKES IT DIFFERENTNo schema Large individual objects... Writing a Report People Read Communicating findings.WHAT TO LEAD WITHThe finding and the recommendation.Not the method.THE... Writing Maintainable SQL Queries others can read.WHAT MAKES SQL UNMAINTAINABLEVery long single queries Deeply nested...
Back

Are you happy with your experience? Leave us a review on Trustpilot.


Trustpilot