Data integration – from collection to delivery Data, in its raw form, originates from a myriad of sources, demanding integration to unlock its full potential. The integration might unfurl in batch sequences or be streamed in real time, aiming for prompt insight derivation. But integration isn’t a standalone task—it demands orchestrated actions to curate, ingest, […]
Understanding the path of data Modern organizations grapple with an avalanche of data from diverse origins. Grasping where this data originates, as well as its consumption patterns, is crucial for assuring its quality and reliability. This is where data lineage emerges as a potent instrument, enabling a clearer overview of data within organizations. Essentially, data […]
We can then see the visual interpretation (Figure 14.1) of which tokens impact the final decision the most: # Sample tweet to explaintweet = “I love using the new feature!So helpful.”# Generate the explanationexp = explainer.explain_instance(tweet, predictor, num_features=5, top_labels=3)exp.show_in_notebook() In this case, num_features determines how many features the explainer should use to describe the prediction. […]
The many facets of ML development As we’ve already touched on in our ML chapters, there are several dimensions to consider when thinking about ML at any level. Let’s recap some of the aspects of ML development as they relate to data stewardship: Beyond training – model deployment and monitoring Ensuring model accuracy doesn’t end […]
Transformative architectural principles In order to maximize the benefits of governance, certain foundational principles must be integrated: Zooming in on architectural dimensions Let’s take another look at our five pillars of governance: Summary In our journey through the realms of data, ML, and architectural governance, we’ve underscored the paramount importance of a cohesive strategy for […]
Introduction to the COMPAS dataset case study In the realm of machine learning, where data drives decision-making, the line between algorithmic precision and ethical fairness often blurs. The COMPAS dataset, a collection of criminal offenders screened in Broward County, Florida, during 2013-2014, serves as a poignant reminder of this intricate dance. While, on the surface, […]
Final thoughts Remember, while we strive to achieve the best model performance, it’s crucial to constantly revisit the fairness aspect. Addressing fairness isn’t a one-time task but rather an iterative process that involves refining the model, re-evaluating fairness metrics, and ensuring that our model decisions are as impartial as possible. Our ultimate aim is to […]
