其他分享
首页 > 其他分享> > keras_2-1

keras_2-1

作者：互联网

1. Agenda

2. Basic operations

  1. tensorborad使用介绍

  2. Constants, Sequences, Variables, Ops

    • constants:

      • normal tensors: e.g. 1-dim,2-dim,more-dim
      • special tensors, e.g. zeros(), zeros_like(), ones(), ones_like(), fill()

    • sequences:

      • Constants as sequences: e.g. lin_space(), range()

      • Randomly Generated Constants: e.g.

        tf.random_normal
tf.truncated_normal
tf.random_uniform
tf.random_shuffle
tf.random_crop
tf.multinomial
tf.random_gamma
        • truncated_normal:
          • 从截断的正态分布中输出随机值。 生成的值服从具有指定均值和标准方差的正态分布，如果生成的值落在（μ-2σ，μ+2σ）之外,则丢弃这个生成的值重新选择。
          • 在正态分布的曲线中，横轴区间（μ-σ，μ+σ）内的面积为68.268949%。
            横轴区间（μ-2σ，μ+2σ）内的面积为95.449974%。
            横轴区间（μ-3σ，μ+3σ）内的面积为99.730020%。
            X落在（μ-3σ，μ+3σ）以外的概率小于千分之三，在实际问题中常认为相应的事件是不会发生的，基本上可以把区间（μ-3σ，μ+3σ）看作是随机变量X实际可能的取值区间，这称之为正态分布的“3σ”原则。
          • 在tf.truncated_normal中如果x的取值在区间（μ-2σ，μ+2σ）之外则重新进行选择。这样保证了生成的值都在均值附近。(it doesn’t create any values more than two standard deviations away from its mean.)
          • tf.set_random_seed(seed)

    • Operations

3. Tensor types

  1. TensorFlow takes Python natives types: boolean, numeric (int, float), strings

  2. tensor包括：0-dim(scalar), 1-dim(vector), 2-dim(matrix), more-dim.

    Single values will be converted to 0-d tensors (or scalars), lists of values will be converted to 1-d tensors (vectors), lists of lists of values will be converted to 2-d tensors (matrices), and so on.

  3. e.g.

    t_0 = 19                                 # scalars are treated like 0-d tensors
tf.zeros_like(t_0)                           # ==> 0
tf.ones_like(t_0)                                # ==> 1

t_1 = [b"apple", b"peach", b"grape"]     # 1-d arrays are treated like 1-d tensors
tf.zeros_like(t_1)                               # ==> [b'' b'' b'']
tf.ones_like(t_1)                                # ==> TypeError: Expected string, got 1 of type 'int' instead.

t_2 = [[True, False, False],
  [False, False, True],
  [False, True, False]]              # 2-d arrays are treated like 2-d tensors

tf.zeros_like(t_2)                               # ==> 3x3 tensor, all elements are False
tf.ones_like(t_2)                                # ==> 3x3 tensor, all elements are True

  4. TensorFlow Data Types

  5. TF vs NP Data Types

    TensorFlow integrates seamlessly with NumPy
tf.int32 == np.int32             # ⇒ True

Can pass numpy types to TensorFlow ops
tf.ones([2, 2], np.float32)  # ⇒ [[1.0 1.0], [1.0 1.0]]

For  tf.Session.run(fetches): if the requested fetch is a Tensor , output will be a NumPy ndarray.
sess = tf.Session()
a = tf.zeros([2, 3], np.int32)
print(type(a))           # ⇒ <class 'tensorflow.python.framework.ops.Tensor'>
a = sess.run(a)
print(type(a))           # ⇒ <class 'numpy.ndarray'>

  6. Use TF DType when possible

    • Python native types: TensorFlow has to infer Python type(转化有计算成本)
    • NumPy arrays: NumPy is not GPU compatible

  7. What’s wrong with constants?、

    • Constants are stored in the graph definition(导致graph太大,且不灵活)
    • This makes loading graphs expensive when constants are big
      • Only use constants for primitive types.
      • Use variables or readers for more data that requires more memory

4. Importing data

  1. Variables：tf.Variable, tf.get_variable

    1. tf.Variable

      tf.Variable holds several ops:

x = tf.Variable(...) 

x.initializer # init op
x.value() # read op
x.assign(...) # write op
x.assign_add(...) # and more

    2. tf.get_variable:

    3. The easiest way is initializing all variables at once:

      • Initializer is an op. You need to execute it within the context of a session
      • sess.run(tf.global_variables_initializer())

    4. Initialize only a subset of variables:

      • sess.run(tf.variables_initializer([a, b])) 仅仅Init了a,b这两个op variables

    5. Initialize a single variable:

      • sess.run(W.initializer)

    6. Eval() a variable (效果类似于W.value()和sess.run(W))

    7. tf.Variable.assign(), tf.assign_add(), tf.assign_sub()

    8. Each session maintains its own copy of variables

    9. Control Dependencies

      # defines which ops should be run first
# your graph g have 5 ops: a, b, c, d, e
    g = tf.get_default_graph()
    with g.control_dependencies([a, b, c]):
        # 'd' and 'e' will only run after 'a', 'b', and 'c' have executed.
        d = ...
        e = …

  2. Placeholder:

    1. Why placeholders?

      We, or our clients, can later supply their own data when they need to execute the computation.

    2. a = tf.placeholder(dtype, shape=None, name=None)
sess.run(c, feed_dict={a: [1, 2, 3]}))    # the tensor a is the key, not the string ‘a’
# shape=None means that tensor of any shape will be accepted as value for placeholder.
# shape=None is easy to construct graphs, but nightmarish for debugging
# shape=None also breaks all following shape inference, which makes many ops not work because they expect certain rank.
# The session will look at the graph, trying to think: hmm, how can I get the value of a, then it computes all the nodes that leads to a.

    3. What if want to feed multiple data points in?

      # You have to do it one at a time
with tf.Session() as sess:
  for a_value in list_of_values_for_a:
  print(sess.run(c, {a: a_value}))

    4. You can feed_dict any feedable tensor.Placeholder is just a way to indicate that something must be fed:

      tf.Graph.is_feedable(tensor) 
# True if and only if tensor is feedable.

    5. Feeding values to TF ops :

      # create operations, tensors, etc (using the default graph)
a = tf.add(2, 5)
b = tf.multiply(a, 3)

with tf.Session() as sess:
  # compute the value of b given a is 15
  sess.run(b, feed_dict={a: 15})              # >> 4

5. Lazy loading

  1. The trap of lazy loading*

  2. What’s lazy loading?

    • Defer creating/initializing an object until it is needed
    • Lazy loading Example: (见cs_2_1的代码test_26,test_27)

  3. Both give the same value of z, What’s the problem? (见cs_2_1的代码test_26,test_27)

    1. normal loding: Node “Add” added once to the graph definition

    2. lazy loding: Node “Add” added 10 times to the graph definition Or as many times as you want to compute z

    3. Imagine you want to compute an op, thousands, or millions of times!

      • Your graph gets bloated Slow to load Expensive to pass around， this is one of the most common TF non-bug bugs I’ve seen on GitHub.
      • 所以要禁止定义匿名op节点

    4. Solution:

      1. Separate definition of ops from computing/running ops

      2. Use Python property to ensure function is also loaded once the first time it is called

        python property

标签：sess,run,like,keras,graph,tf,tensors
来源： https://www.cnblogs.com/LS1314/p/10366203.html