Business intelligence

2 posts
Data Fabric Or Data Fabrication?



Earlier in my career, as a data analyst, the biggest area of frustration for my marketing clients was the lag time between their requests and the business insight. The lag was extended if the required data had to be ingested from a new source. The old ETL framework and associated data management tasks created long waits for impactful insights to reach marketing tactics.

Over the course of my career, despite advances in data strategies and infrastructure, similar questions remain:

  1. “Why can’t I get my analytics sooner?”
  2. “Why can’t I access my own data”
  3. “Why does it take so long to source new data that we need for analytics?”

A recent concept that seeks to relieve the stress associated with these questions is the data fabric. According to Gartner, a data fabric is...

“An architecture pattern that informs and automates the design, integration, and deployment of data objects regardless of deployment platforms and architectural approaches.

"It utilizes continuous analytics and AI/ML over all metadata assets to provide actionable insights and recommendations on data management, integration design, and deployment patterns.

"This results in faster, informed, and (in some cases) complete automation of access and sharing.”

Quite a long multi-layer definition, but based on this promise of data fabrics, data analytics teams can now:

  1. Avoid having to move data
  2. Have better access controls and democratize access
  3. Provide faster, more automated analytics through AI

Business managers often think about these issues and their impact on their business:

Speedier time to insight (shorter cycle times): No more ETL, or ELT, or other time-intensive data management tasks associated with data analytics. AI/ML tools will automate data tasks and data virtualization will allow analysts to go directly to the source elements avoiding time-intensive access and exploration—with the exciting possibility of eliminating extract tools, schedulers, and more.

Expense savings: By going directly to the source, no intermediary platforms, systems, or repositories will be needed. Therefore platform/tool expenses and recurrent maintenance expenses would be saved.

Why Move Data (When You Don’t Have To!)


Data fabric, data transmission

The problem many firms face is the storage of their data in many silos and systems—clouds (lakehouses), warehouses, source systems, ODSs, CRM systems, marketing marts, and in some cases even legacy SAS files. Each of these silos introduce workflows that are necessary but increase time-to-insight. How can organizations use all their data efficiently?

The idea behind the data fabric is to break down data silos and get data into the hands of the users. The data fabric is a tapestry connecting data across all platforms to users, creating efficiencies by not having to move data. Data automation capabilities within the fabric further drive efficiencies by providing accessible, quality data for use by AI and other tasks.

One of the key enablers of a data fabric is a virtualization layer that provisions data needs directly (from transaction and operational systems) without moving or copying data. Data tasks like sourcing, extracting, cleaning/transforming, etc. are all automated. The fabric also helps manage the lifecycle of data, for example, by governing your data using active metadata to enforce the policies, including access, compliance, and quality.

In summary, data fabrics are still evolving—the architecture has merit, the concept seems more developed than that of the data mesh (which is much more of a theory), and foundational tech capabilities are well-enabled. I see fabrics as having great promise, but more as an architecture strategy with a focus on less data movement, fewer silos, more access to data, and faster analytics. Will fabrics eliminate the need to move data completely? I don’t think so (at least in the near term), as there will always be a need to move and replicate some data. Would love to hear from the vendors in the space on this post as well.

Is it data fabric or data fabrication? Only further development (and time) will tell, but I’m betting on the fabric’s promise!

I’m also betting on you, my readers—I bet that a discussion with you on this topic will surface many challenges and opportunities for data fabrics. What would you like to learn more about? What has been your experience with the data fabric? Is it fabric or fabrication? Please reply to this post with your comments so we can all dive deeper into the strategic, tactical, and business implications of this fascinating area.

Acquiring Job Advance Your Career Career Changing Careers and Employment Education Internship

Data Fabric Or Data Fabrication?

Data Never Sleeps. How Actionable Is Your Reporting?



The business seems to be doing better, but you have reporting to show how well it's actually doing? You want to collect data and turn it into information. This allows the business to make decisions based on actionable reporting. How much business intelligence (BI) does your organization have?

COTS Applications For Actionable Reporting


Most COTS (commercial-off-the-shelf) applications will have standard/canned reports. These are the “common” reports used by customers. If you’re implementing a new system and there are many standard reports, start with 10 top reports so you don’t overwhelm the end users. If you’re not sure what these are, ask the vendor to recommend them (especially if you’re changing your process and following the vendor’s best practice). Then the end users can utilize other standard reports as they get more comfortable and familiar with the application.

For example, if you just purchased a new phone system for the call center, find the reports that detail measurements such as the number of calls, dropped calls, wait time, peak times, etc.

Identify and define key performance indicators (KPIs), which are measurable values to analyze for better decision making. KPIs may be related to a specific function, line of business, or group of individuals. You can take a group of KPIs and create a dashboard. For example, some KPIs related to the help desk are:

  • Tickets by status (e.g., open, on hold, completed, canceled)
  • Tickets assigned by technician
  • Average resolution time
  • Rate of first-time resolutions
  • Customer satisfaction

Once you’re comfortably leveraging the standard reports, you may want to ask the vendor (or your team’s developer/programmer) to customize standard reports or create custom reports with additional specific information you’re looking for. This may involve combining data from multiple tables within the application.

Two typical ways to serve these reports: 1) “push” reports to the end users on a scheduled frequency such as via email daily; or 2) have the end user “pull” reports which gives them the ability to access the reports as needed.

But reporting isn’t limited to COTS applications. You can create remarkable reports using Excel such as PivotTables, Pivot Charts, or Slicers.

Other Tips & Tools For Actionable Reporting


Another phenomenal visualization tool that is part of Office 365 is Power BI. It’s part of the Microsoft family (similar to Excel) so it’s straightforward to use. You can create a dashboard visualizing year-over-year comparisons, market segmentations, drill downs, etc.

If you’re new to Power BI, you can download the free desktop version (which admittedly does have limitations). If you find that Power BI does meet your basic reporting needs and you want more, then you can upgrade to the Pro license.

If you have enormous amounts of data, you may want to consider creating a data lake (raw data) or data warehouse (structured data). You can pull in data from multiple disparate systems into one central repository. There are ETL (extract transform load) tools to load the data into the data warehouse. You can cube the data so that the end users can roll up, drill down, and slice the data, do “what if” scenarios, etc.

TIP: If you allow end users to create their own reports, it’s critical that they understand the various data fields. Are you able to give them some type of data dictionary, or are they able to certify the results? It would be disastrous if they selected the wrong date or revenue field which produced incorrect results and sent the report to the board of directors.

Other reporting considerations:

1. Have a separate reporting database so that you don’t affect the production database. You don’t want an end user to kick off a complex query inadvertently causing production response to come to a crawl or even freeze.

  • If you do set up a reporting database you need to decide how frequently the data will be updated—real time, once in the evening, etc.

2. Data needs to be secured appropriately (security). Who should have access to specific data fields should be determined by the data owner. For example, only a small handful of people should be able to access salary/payroll information.

3. Data should be classified such as public, internal, confidential, or restricted. Access should be controlled as appropriate.

4. Create a customer portal (extranet) to empower your customers (also partners and vendors) so that they can pull their own reports and information.

There is a quote by Lois Horowitz: “Not having the information you need when you need it leaves you wanting. Not knowing where to look for that information leaves you powerless. In a society where information is king, none of us can afford that.”

For more information on actionable reporting, follow me on LinkedIn!

Acquiring Job Advance Your Career Career Changing Careers and Employment Education Internship

Data Never Sleeps. How Actionable Is Your …