Book Image

Data Analysis with Python

By : David Taieb
Book Image

Data Analysis with Python

By: David Taieb

Overview of this book

Data Analysis with Python offers a modern approach to data analysis so that you can work with the latest and most powerful Python tools, AI techniques, and open source libraries. Industry expert David Taieb shows you how to bridge data science with the power of programming and algorithms in Python. You'll be working with complex algorithms, and cutting-edge AI in your data analysis. Learn how to analyze data with hands-on examples using Python-based tools and Jupyter Notebook. You'll find the right balance of theory and practice, with extensive code files that you can integrate right into your own data projects. Explore the power of this approach to data analysis by then working with it across key industry case studies. Four fascinating and full projects connect you to the most critical data analysis challenges you’re likely to meet in today. The first of these is an image recognition application with TensorFlow – embracing the importance today of AI in your data analysis. The second industry project analyses social media trends, exploring big data issues and AI approaches to natural language processing. The third case study is a financial portfolio analysis application that engages you with time series analysis - pivotal to many data science applications today. The fourth industry use case dives you into graph algorithms and the power of programming in modern data science. You'll wrap up with a thoughtful look at the future of data science and how it will harness the power of algorithms and artificial intelligence.
Table of Contents (16 chapters)
Data Analysis with Python
Contributors
Preface
Other Books You May Enjoy
3
Accelerate your Data Analysis with Python Libraries
Index

Custom HTML attributes


These can be used with any regular HTML elements to configure kernel requests. The PixieApp framework can trigger these requests when the element receives a click or change event, or right after the HTML fragment has completed loading.

  • pd_options: List of key-value pairs that define transient states for the kernel request, according to the following format: pd_options=”key1=value1;key2=value2;...”. When used in combination with the pd_entity attribute, the pd_options attribute invokes the PixieDust display() API. In this case, you can get the values from the metadata of a separate Notebook cell in which you have used the display() API. When using pd_options in display() mode, it is recommended for convenience, to use the JSON notation of pd_options by creating a child element called <pd_options> and include the JSON values as text.

    Example with pd_options as child element invoking display():

    <div pd_entity>
        <pd_options>
            {
                “mapboxtoken”: “XXXXX”,
                “chartsize”: “90”,
                “aggregation”: “SUM”,
                “rowCount”: “500”,
                “handlerId”: “mapView”,
                “rendererId”: “mapbox”,
                “valueFields”: “IncidntNum”,
                “keyFields”: “X,Y”,
                “basemap”: “light-v9”
            }
        </pd_options>
    </div>

    Example with pd_options as HTML attribute:

    <!-- Invoke a route that displays a chart -->
    <button type=”submit” pd_options=”showChart=true” pd_target=”chart{{prefix}}”>
        Show Chart
    </button>
  • pd_entity: Used only to invoke the display() API on specific data. Must be used in combination with pd_options where key-value pairs will be used as arguments to display(). If no value is specified for the pd_entity attribute, then it is assumed to be the entity passed to the run method that starts the PixieApp. The pd_entity value can be either a variable defined in the Notebook or a field of the PixieApp (for example, pd_entity=”df”), or a field to an object using the dot notation (for example, pd_entity=”obj_instance.df”).

  • pd_target: By default, the output of a kernel request is injected in the overall output cell or dialog (if you use runInDialog=”true” as an argument to the run method). However, you can use pd_target=”elementId” to specify a target element that will receive the output. (Note that the elementId must exist in the current view.)

    Example:

    <div id=”chart{{prefix}}”>
    <button type=”submit” pd_options=”showChart=true” pd_target=”chart{{prefix}}”>
        Show Chart
    </button>
    </div>
  • pd_script: This invokes arbitrary Python code as part of the kernel request. This can be used in combination with other attributes like pd_entity and pd_options. It’s important to note that the Python indentation rules (https://docs.python.org/2.0/ref/indentation.html) must be respected to avoid a runtime error.

    If the Python code contains multiple lines, it is recommended to use pd_script as a child element and store the code as text.

    Example:

    <!-- Invoke a method to load a dataframe before visualizing it -->
    <div id=”chart{{prefix}}”>
    <button type=”submit”
        pd_entity=”df”
        pd_script=”self.df = self.load_df()”
        pd_options=”handlerId=dataframe”
        pd_target=”chart{{prefix}}”>
        Show Chart
    </button>
    </div>
  • pd_app: This dynamically invokes a separate PixieApp by its fully qualified class name. The pd_options attribute can be used to pass route arguments to invoke a specific route of the PixieApp.

    Example:

    <div pd_render_onload
         pd_option=”show_route_X=true”
         pd_app=”some.package.RemoteApp”>
    </div>
  • pd_render_onload: This should be used to trigger a kernel request upon loading, as opposed to when a user clicks on an element or when a change event occurs. The pd_render_onload attribute can be combined with any other attribute that defines the request, like pd_options or pd_script. Note that this attribute should only be used with a div element.

    Example:

    <div pd_render_onload>
        <pd_script>
    print(‘hello world rendered on load’)
        </pd_script>
    </div>
  • pd_refresh: This is used to force the HTML element to execute a kernel request even if no event (click or change event) has occurred. If no value is specified, then the current element is refreshed, otherwise, the element with the ID specified in the value will be refreshed.

    Example:

    <!-- Update state before refreshing a chart -->
    <button type=”submit”
        pd_script=”self.show_line_chart()”
        pd_refresh=”chart{{prefix}}”>
        Show line chart
    </button>
  • pd_event_payload: This emits a PixieApp event with the specified payload content. This attribute follows the same rules as pd_options:

    • Each key-value pair must be encoded using the key=value notation

    • The event will be triggered on a click or a change event

    • Support for $val() directive to dynamically inject user entered input

    • Use <pd_event_payload> child to enter raw JSON.

      <button type=”submit” pd_event_payload=”type=topicA;message=Button clicked”>
          Send event A
      </button>
      <button type=”submit”>
          <pd_event_payload>
          {
              “type”:”topicA”,
              “message”:”Button Clicked”
          }
          </pd_event_payload>
          Send event A
      </button>

    Example:

  • pd_event_handler: Subscribers can listen to an event by declaring a <pd_event_handler> child element which can accept any of the PixieApp kernel execution attributes like pd_options and pd_script. This element must use the pd_source attribute to filter which events they want to process. The pd_source attribute can contain one of the following values:

    • targetDivId: Only events originating from the element with the specified ID will be accepted

    • type: Only events with the specified type will be accepted

      <div class=”col-sm-6” id=”listenerA{{prefix}}”>
          Listening to button event
          <pd_event_handler
              pd_source=”topicA”
              pd_script=”print(eventInfo)”
              pd_target=”listenerA{{prefix}}”>
          </pd_event_handler>
      </div>

    Example:

    Note

    You can find the code file here:

    https://github.com/DTAIEB/Thoughtful-Data-Science/blob/master/chapter%205/sampleCode37.html

    Note: Using * for pd_source denotes that all events will be accepted.

  • pd_refresh_rate: This is used to repeat the execution of an element at a specified interval expressed in milliseconds. This is useful for when you want to poll the state of a particular variable and show the result in the UI.

    Example:

    <div pd_refresh_rate=”3000”
        pd_script=”print(self.get_status())”>
    </div>